ORCA benchmark

Rank

Submission Title

Model

URL

Orca Score

Details

Created On

Show Results

Huawei Noah Ark Lab MTL

SABER

77.09

2024-10-20 10:13:14

Submission (Huawei Noah Ark Lab MTL) Model (SABER) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	73.06	ANS Stance	91.3	Aqmar NER	83.67	Arabic NER Corpus	91.24
Baly Stance	50.09	Emotion Regression	77.33	Machine Generation	90.13	MQ2Q	95.82
MSA Part Of Speech	59.32	Question Answering	62.81	STS	78.82	Topic	94.14
Word Sense Disambiguation	76.54	XLNI	71.06

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	82.23	Adult	90.8	Age	45.87	Dangerous	67.86
Dialect at Binary Level	88.43	Dialect at Country Level	40.69	Dialect at Region Level	71.81	Dialect Part Of Speech	94.78
Emotion	70.14	Gender	65.61	Hate Speech	85.81	Irony	85.74
Offensive	92.97	Sarcasm	77.34	Sentiment Analysis	80.18

Huawei Noah Ark Lab MTL

JABERv2

76.14

2024-11-16 13:46:28

Submission (Huawei Noah Ark Lab MTL) Model (JABERv2) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	69.44	ANS Stance	90.77	Aqmar NER	81.23	Arabic NER Corpus	90.4
Baly Stance	46.15	Emotion Regression	76.97	Machine Generation	88.8	MQ2Q	98.83
MSA Part Of Speech	59.93	Question Answering	63.24	STS	77.44	Topic	93.53
Word Sense Disambiguation	74.89	XLNI	71.37

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	78.15	Adult	90.28	Age	46.86	Dangerous	67.8
Dialect at Binary Level	87.39	Dialect at Country Level	37.95	Dialect at Region Level	69.64	Dialect Part Of Speech	93.76
Emotion	69.59	Gender	67.54	Hate Speech	86.84	Irony	82.52
Offensive	92.27	Sarcasm	74.26	Sentiment Analysis	80.24

Huawei Noah Ark Lab MTL

AT5Bv2

75.52

2024-11-16 13:46:04

Submission (Huawei Noah Ark Lab MTL) Model (AT5Bv2) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	69.04	ANS Stance	93.3	Aqmar NER	82.5	Arabic NER Corpus	90.58
Baly Stance	46.66	Emotion Regression	75.34	Machine Generation	87.44	MQ2Q	96.15
MSA Part Of Speech	57.5	Question Answering	62.81	STS	75.76	Topic	94.23
Word Sense Disambiguation	74.98	XLNI	71.97

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	76.63	Adult	90.54	Age	45.44	Dangerous	66.54
Dialect at Binary Level	87.94	Dialect at Country Level	37.59	Dialect at Region Level	66.29	Dialect Part Of Speech	93.66
Emotion	65.7	Gender	67.37	Hate Speech	84.02	Irony	83.89
Offensive	91.68	Sarcasm	74.76	Sentiment Analysis	79.73

Huawei Noah Ark Lab MTL

JABER

75.48

2024-03-28 08:55:21

Submission (Huawei Noah Ark Lab MTL) Model (JABER) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	70.78	ANS Stance	92.33	Aqmar NER	81.55	Arabic NER Corpus	90.68
Baly Stance	48.81	Emotion Regression	71.47	Machine Generation	89.35	MQ2Q	96.24
MSA Part Of Speech	57.54	Question Answering	62.81	STS	75.08	Topic	93.68
Word Sense Disambiguation	75.41	XLNI	71.27

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	81.26	Adult	90.3	Age	43.96	Dangerous	66.83
Dialect at Binary Level	87.92	Dialect at Country Level	38.06	Dialect at Region Level	65.53	Dialect Part Of Speech	93.44
Emotion	67.2	Gender	64.98	Hate Speech	83.42	Irony	84.76
Offensive	89.72	Sarcasm	74.22	Sentiment Analysis	80.23

Baselines

ARBERTv2

74.02

2023-10-16 14:48:17

Submission (Baselines) Model (ARBERTv2) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	67.38	ANS Stance	91.02	Aqmar NER	81.7	Arabic NER Corpus	90.83
Baly Stance	49.34	Emotion Regression	67.73	Machine Generation	87.94	MQ2Q	95.99
MSA Part Of Speech	52.55	Question Answering	61.56	STS	71.9	Topic	93.96
Word Sense Disambiguation	71.01	XLNI	68.17

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	75.99	Adult	89.67	Age	45.57	Dangerous	64.96
Dialect at Binary Level	86.91	Dialect at Country Level	35.69	Dialect at Region Level	65.21	Dialect Part Of Speech	93.92
Emotion	64.81	Gender	63.18	Hate Speech	82.26	Irony	83.83
Offensive	89.55	Sarcasm	74.16	Sentiment Analysis	78.6

Baselines

AraBERTv02

74.01

2023-10-16 14:48:17

Submission (Baselines) Model (AraBERTv02) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	69.34	ANS Stance	93.23	Aqmar NER	79.2	Arabic NER Corpus	90.85
Baly Stance	46.63	Emotion Regression	67.64	Machine Generation	89.82	MQ2Q	94.01
MSA Part Of Speech	37.84	Question Answering	61.5	STS	75.4	Topic	94.58
Word Sense Disambiguation	76.68	XLNI	67.93

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	78.06	Adult	89.9	Age	44.33	Dangerous	62.6
Dialect at Binary Level	87.01	Dialect at Country Level	36.33	Dialect at Region Level	67.5	Dialect Part Of Speech	93.79
Emotion	64.89	Gender	64.37	Hate Speech	82.39	Irony	84.51
Offensive	90.73	Sarcasm	74.06	Sentiment Analysis	80.5

Baselines

ARBERT

73.77

2023-10-16 14:48:17

Submission (Baselines) Model (ARBERT) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	70.25	ANS Stance	91.77	Aqmar NER	80.72	Arabic NER Corpus	90.27
Baly Stance	45.63	Emotion Regression	65.37	Machine Generation	88.35	MQ2Q	95.15
MSA Part Of Speech	51.39	Question Answering	57.65	STS	73	Topic	94.17
Word Sense Disambiguation	72.94	XLNI	67.22

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	76.53	Adult	89.7	Age	44.76	Dangerous	67.32
Dialect at Binary Level	86.48	Dialect at Country Level	35.26	Dialect at Region Level	67.71	Dialect Part Of Speech	92.92
Emotion	63.6	Gender	64.38	Hate Speech	82.41	Irony	83.53
Offensive	89.28	Sarcasm	73.11	Sentiment Analysis	77.78

Baselines

AraBERTv02-Twitter

73.77

2023-10-16 14:48:17

Submission (Baselines) Model (AraBERTv02-Twitter) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	71.53	ANS Stance	90.17	Aqmar NER	77.66	Arabic NER Corpus	90.03
Baly Stance	37.12	Emotion Regression	70.31	Machine Generation	87.96	MQ2Q	94.82
MSA Part Of Speech	41.28	Question Answering	57.65	STS	70.5	Topic	94.02
Word Sense Disambiguation	72.92	XLNI	66.67

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	76.87	Adult	90.18	Age	47.26	Dangerous	65.66
Dialect at Binary Level	87.76	Dialect at Country Level	36.5	Dialect at Region Level	66.34	Dialect Part Of Speech	94.7
Emotion	66.7	Gender	65.65	Hate Speech	85.3	Irony	84.62
Offensive	91.89	Sarcasm	76.83	Sentiment Analysis	80.86

Huawei Noah Ark Lab MTL

JABERv2-6L

73.72

2024-11-16 13:46:00

Submission (Huawei Noah Ark Lab MTL) Model (JABERv2-6L) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	69.16	ANS Stance	87.65	Aqmar NER	81.11	Arabic NER Corpus	90.63
Baly Stance	42.22	Emotion Regression	74.17	Machine Generation	87.41	MQ2Q	94.73
MSA Part Of Speech	51.12	Question Answering	56.57	STS	71.92	Topic	93.57
Word Sense Disambiguation	72.82	XLNI	67.1

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	77.05	Adult	90.09	Age	45.05	Dangerous	63.72
Dialect at Binary Level	87.66	Dialect at Country Level	36.07	Dialect at Region Level	65.44	Dialect Part Of Speech	93.58
Emotion	63.96	Gender	66.27	Hate Speech	81.77	Irony	83.76
Offensive	90.41	Sarcasm	72.75	Sentiment Analysis	80.15

Baselines

GigaBERT-v3

73.65

2023-10-16 14:48:17

Submission (Baselines) Model (GigaBERT-v3) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	69.08	ANS Stance	86.9	Aqmar NER	77.32	Arabic NER Corpus	89.77
Baly Stance	49.7	Emotion Regression	64.38	Machine Generation	87.59	MQ2Q	96.07
MSA Part Of Speech	62.58	Question Answering	57.74	STS	72.21	Topic	94.12
Word Sense Disambiguation	74.61	XLNI	67.14

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	75.01	Adult	89.21	Age	44.29	Dangerous	64.82
Dialect at Binary Level	86.57	Dialect at Country Level	33.99	Dialect at Region Level	68.13	Dialect Part Of Speech	93.24
Emotion	61.92	Gender	63.78	Hate Speech	81.73	Irony	81.33
Offensive	89.39	Sarcasm	73.34	Sentiment Analysis	78.69

Baselines

GigaBERT-v4

73.54

2023-10-16 14:48:17

Submission (Baselines) Model (GigaBERT-v4) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	68.81	ANS Stance	85.31	Aqmar NER	74.97	Arabic NER Corpus	89.52
Baly Stance	49.21	Emotion Regression	66.57	Machine Generation	87.43	MQ2Q	94.64
MSA Part Of Speech	63.89	Question Answering	58.71	STS	73.68	Topic	94.05
Word Sense Disambiguation	74.31	XLNI	66.53

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	77.34	Adult	89.35	Age	43.83	Dangerous	62.83
Dialect at Binary Level	86.13	Dialect at Country Level	33.96	Dialect at Region Level	69.22	Dialect Part Of Speech	92.57
Emotion	61.92	Gender	63.97	Hate Speech	82.25	Irony	80.91
Offensive	88.76	Sarcasm	72.02	Sentiment Analysis	78.68

Baselines

CAMeLBERT-MSA

73.29

2023-10-16 14:48:17

Submission (Baselines) Model (CAMeLBERT-MSA) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	70.32	ANS Stance	89.21	Aqmar NER	78.54	Arabic NER Corpus	90.71
Baly Stance	49.67	Emotion Regression	65.75	Machine Generation	88.62	MQ2Q	95.23
MSA Part Of Speech	32.89	Question Answering	57.46	STS	74.42	Topic	94.38
Word Sense Disambiguation	75.86	XLNI	66.02

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	76.98	Adult	90.09	Age	45.97	Dangerous	65.31
Dialect at Binary Level	87.4	Dialect at Country Level	35.18	Dialect at Region Level	65.05	Dialect Part Of Speech	93.95
Emotion	63.85	Gender	65.19	Hate Speech	79.68	Irony	83.07
Offensive	89.32	Sarcasm	75.18	Sentiment Analysis	79.75

Huawei Noah Ark Lab MTL

AT5Sv2

72.68

2024-11-16 13:45:55

Submission (Huawei Noah Ark Lab MTL) Model (AT5Sv2) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	70.23	ANS Stance	84.15	Aqmar NER	77.38	Arabic NER Corpus	90.81
Baly Stance	49.16	Emotion Regression	69.59	Machine Generation	86.55	MQ2Q	95.07
MSA Part Of Speech	52.89	Question Answering	54.7	STS	74.17	Topic	94.01
Word Sense Disambiguation	70.39	XLNI	64.84

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	72.2	Adult	89.42	Age	44.16	Dangerous	61.81
Dialect at Binary Level	87.7	Dialect at Country Level	32.17	Dialect at Region Level	65.75	Dialect Part Of Speech	93.31
Emotion	61.02	Gender	65.67	Hate Speech	78.13	Irony	82.65
Offensive	89.84	Sarcasm	72.14	Sentiment Analysis	77.9

Huawei Noah Ark Lab MTL

AT5B

72.39

2024-11-16 13:45:50

Submission (Huawei Noah Ark Lab MTL) Model (AT5B) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	64.78	ANS Stance	80.69	Aqmar NER	78.73	Arabic NER Corpus	91.44
Baly Stance	44.05	Emotion Regression	68.69	Machine Generation	88.33	MQ2Q	95.32
MSA Part Of Speech	30.51	Question Answering	61.95	STS	72.9	Topic	94.37
Word Sense Disambiguation	74.29	XLNI	71.66

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	73.49	Adult	90.31	Age	43.44	Dangerous	59.37
Dialect at Binary Level	86.77	Dialect at Country Level	34.2	Dialect at Region Level	65.61	Dialect Part Of Speech	93.27
Emotion	62.9	Gender	63.31	Hate Speech	80.6	Irony	82.24
Offensive	90.76	Sarcasm	76.45	Sentiment Analysis	78.98

Baselines

AraELECTRA

72.33

2023-10-16 14:48:17

Submission (Baselines) Model (AraELECTRA ) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	70.35	ANS Stance	92.33	Aqmar NER	73.43	Arabic NER Corpus	90.17
Baly Stance	41.32	Emotion Regression	61.51	Machine Generation	90.66	MQ2Q	96.65
MSA Part Of Speech	26.61	Question Answering	60.7	STS	76.01	Topic	94.11
Word Sense Disambiguation	73.54	XLNI	70.2

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	76.22	Adult	89.65	Age	42.02	Dangerous	67.13
Dialect at Binary Level	87.21	Dialect at Country Level	27	Dialect at Region Level	64.46	Dialect Part Of Speech	93.554
Emotion	60.98	Gender	64.24	Hate Speech	81.79	Irony	81.56
Offensive	89.4	Sarcasm	74.81	Sentiment Analysis	79.56

Baselines

CAMeLBERT-MIX

72.27

2023-10-16 14:48:17

Submission (Baselines) Model (CAMeLBERT-MIX) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	68.96	ANS Stance	82.49	Aqmar NER	76.84	Arabic NER Corpus	89.27
Baly Stance	48.94	Emotion Regression	66.73	Machine Generation	86.73	MQ2Q	94.99
MSA Part Of Speech	27.7	Question Answering	53.98	STS	70.61	Topic	93.72
Word Sense Disambiguation	74.12	XLNI	63.82

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	75.49	Adult	90.33	Age	45.89	Dangerous	64.03
Dialect at Binary Level	86.79	Dialect at Country Level	33.9	Dialect at Region Level	65.59	Dialect Part Of Speech	94.37
Emotion	65.25	Gender	66.38	Hate Speech	81.99	Irony	84.01
Offensive	90.05	Sarcasm	74.17	Sentiment Analysis	79.51

Baselines

MARBERTv2

72.04

2023-10-16 14:48:17

Submission (Baselines) Model (MARBERTv2) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	67.83	ANS Stance	80.57	Aqmar NER	74.22	Arabic NER Corpus	87.86
Baly Stance	36.52	Emotion Regression	74.27	Machine Generation	86.69	MQ2Q	95.49
MSA Part Of Speech	55.45	Question Answering	57.9	STS	66.12	Topic	93.9
Word Sense Disambiguation	33.29	XLNI	62.22

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	78.03	Adult	90.97	Age	46.24	Dangerous	67.11
Dialect at Binary Level	86.92	Dialect at Country Level	36.06	Dialect at Region Level	66.32	Dialect Part Of Speech	93.38
Emotion	70.82	Gender	67.64	Hate Speech	83.54	Irony	83.09
Offensive	92.23	Sarcasm	76.19	Sentiment Analysis	80.83

Baselines

Arabic BERT

71.8

2023-10-16 14:48:17

Submission (Baselines) Model (Arabic BERT) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	66.29	ANS Stance	82.86	Aqmar NER	79.5	Arabic NER Corpus	88.77
Baly Stance	49.22	Emotion Regression	62.77	Machine Generation	86.28	MQ2Q	93.89
MSA Part Of Speech	42.4	Question Answering	55.94	STS	73.56	Topic	93.86
Word Sense Disambiguation	69.76	XLNI	62.15

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	72.28	Adult	89.74	Age	45.23	Dangerous	64.56
Dialect at Binary Level	85.94	Dialect at Country Level	32.99	Dialect at Region Level	63.98	Dialect Part Of Speech	91.66
Emotion	62.2	Gender	64.35	Hate Speech	79.77	Irony	83.03
Offensive	87.52	Sarcasm	73.11	Sentiment Analysis	78.46

Baselines

AraBERTv01

71.39

2023-10-16 14:48:17

Submission (Baselines) Model (AraBERTv01) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	69.74	ANS Stance	87.59	Aqmar NER	77.34	Arabic NER Corpus	90.68
Baly Stance	51.19	Emotion Regression	62.02	Machine Generation	87.45	MQ2Q	94.82
MSA Part Of Speech	34.65	Question Answering	56.79	STS	71.27	Topic	94.31
Word Sense Disambiguation	51.72	XLNI	64.69

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	75.92	Adult	88.97	Age	44.1	Dangerous	64.72
Dialect at Binary Level	86.64	Dialect at Country Level	31.49	Dialect at Region Level	64.63	Dialect Part Of Speech	91.65
Emotion	60.6	Gender	62.41	Hate Speech	82.21	Irony	83.63
Offensive	87.5	Sarcasm	73.46	Sentiment Analysis	78.45

Huawei Noah Ark Lab MTL

AT5S

71.02

2024-11-16 13:45:48

Submission (Huawei Noah Ark Lab MTL) Model (AT5S) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	66.04	ANS Stance	85.02	Aqmar NER	79.76	Arabic NER Corpus	89.71
Baly Stance	45.09	Emotion Regression	62.47	Machine Generation	86.59	MQ2Q	94.81
MSA Part Of Speech	25.81	Question Answering	54.66	STS	73.25	Topic	93.9
Word Sense Disambiguation	71.27	XLNI	65.72

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	73.31	Adult	89.66	Age	42.44	Dangerous	61.06
Dialect at Binary Level	87.18	Dialect at Country Level	31.22	Dialect at Region Level	64.97	Dialect Part Of Speech	92.27
Emotion	59.78	Gender	62.07	Hate Speech	82.52	Irony	81.39
Offensive	87.99	Sarcasm	71.84	Sentiment Analysis	77.76

Baselines

MARBERT

70.61

2023-10-16 14:48:18

Submission (Baselines) Model (MARBERT) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	67.91	ANS Stance	73.82	Aqmar NER	74.57	Arabic NER Corpus	86.59
Baly Stance	29.07	Emotion Regression	73.96	Machine Generation	85.14	MQ2Q	93.82
MSA Part Of Speech	55.02	Question Answering	49.35	STS	63.48	Topic	93.53
Word Sense Disambiguation	33.29	XLNI	60.5

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	78.36	Adult	90.76	Age	47.11	Dangerous	66.2
Dialect at Binary Level	86.78	Dialect at Country Level	35.83	Dialect at Region Level	66.08	Dialect Part Of Speech	94.14
Emotion	68.85	Gender	66.65	Hate Speech	81.33	Irony	83.27
Offensive	91.84	Sarcasm	74.74	Sentiment Analysis	79.08

Baselines

QARiB

69.79

2023-10-16 14:48:18

Submission (Baselines) Model (QARiB) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	69.2	ANS Stance	50.2	Aqmar NER	73.72	Arabic NER Corpus	87.5
Baly Stance	31.58	Emotion Regression	71.91	Machine Generation	86.35	MQ2Q	94.06
MSA Part Of Speech	41.36	Question Answering	45.27	STS	41.15	Topic	93.32
Word Sense Disambiguation	70.13	XLNI	55.67

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	79.66	Adult	90.89	Age	46.35	Dangerous	66.25
Dialect at Binary Level	87.85	Dialect at Country Level	34.36	Dialect at Region Level	66.71	Dialect Part Of Speech	93.95
Emotion	68.03	Gender	66.64	Hate Speech	83.88	Irony	85.16
Offensive	91.17	Sarcasm	75.82	Sentiment Analysis	80.33

Baselines

CAMeLBERT-CA

68.17

2023-10-16 14:48:18

Submission (Baselines) Model (CAMeLBERT-CA) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	63.06	ANS Stance	71.81	Aqmar NER	70.71	Arabic NER Corpus	82.94
Baly Stance	48.85	Emotion Regression	48.46	Machine Generation	83.35	MQ2Q	94.23
MSA Part Of Speech	17.04	Question Answering	52.11	STS	70.13	Topic	93.41
Word Sense Disambiguation	75.68	XLNI	61.18

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	74.43	Adult	88.68	Age	43.29	Dangerous	61.97
Dialect at Binary Level	84.58	Dialect at Country Level	27.05	Dialect at Region Level	63.36	Dialect Part Of Speech	92.05
Emotion	59.66	Gender	63.37	Hate Speech	74.1	Irony	79.68
Offensive	86.52	Sarcasm	69.94	Sentiment Analysis	76.76

Baselines

CAMeLBERT-DA

67.21

2023-10-16 14:48:18

Submission (Baselines) Model (CAMeLBERT-DA) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	65.66	ANS Stance	46.01	Aqmar NER	73.77	Arabic NER Corpus	83.61
Baly Stance	30.58	Emotion Regression	64.34	Machine Generation	83.17	MQ2Q	90.63
MSA Part Of Speech	10.37	Question Answering	30.91	STS	71.23	Topic	93.18
Word Sense Disambiguation	65.18	XLNI	54.29

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	73.57	Adult	90.76	Age	45.29	Dangerous	66.92
Dialect at Binary Level	86.64	Dialect at Country Level	30.83	Dialect at Region Level	68.55	Dialect Part Of Speech	94.14
Emotion	64.8	Gender	64.25	Hate Speech	83.38	Irony	81.91
Offensive	90.44	Sarcasm	72.57	Sentiment Analysis	77.8

Baselines

Arabic-ALBERT

58.14

2023-07-10 15:23:02

Submission (Baselines) Model (Arabic-ALBERT) Results

Modern Standard Arabic (MSA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
ANS Claim	65.88	ANS Stance	80.02	Aqmar NER	73.15	Arabic NER Corpus	86.54
Baly Stance	47.19	Emotion Regression	45.72	Machine Generation	83.91	MQ2Q	94.56
MSA Part Of Speech	32.1	Question Answering	53.89	STS	66.75	Topic	93.27
Word Sense Disambiguation	47.19	XLNI	61.62

Arabic Dialect (DA) Tasks
Task	Score	Task	Score	Task	Score	Task	Score
Abusive	67.68	Adult	88.88	Age	43.61	Dangerous	63.41
Dialect at Binary Level	84.45	Dialect at Country Level	28.25	Dialect at Region Level	62.87	Dialect Part Of Speech	91.66
Emotion	55.22	Gender	63.5	Hate Speech	74.26	Irony	79.05
Offensive	85.26	Sarcasm	71.92	Sentiment Analysis	76.46