Mashq and Machine

Swidan and Hermans (2023) presented a framework of 12 “aspects of programming languages that can be translated to non-English [contexts]”. ArabicBASIC’s ambitious goal is to comply with this framework 100%. That’s one success criteria for this project.

Another success criteria is that ArabicBASIC will strictly conform to the observed linguistic situation in the Arabic-speaking world, which presents several technical challenges. One is accommodating diglossia amongst Arabic speakers in the Lexer’s workings, possibly even in the parser and interpreter. I’m exploring a technique which I hope will make it elegant, because programmers are lazy :-) and I hate inelegant solutions.

Nevertheless, I’ll satisfy this criterion using several tools:

• Unicode’s CLDR research
• Selecting keywords by examining them in different Arabic corpora to confirm their usage semantics. This has already borne fruit and led to some keyword changes. This will be a large topic and saved for a future post.

I am not aware of hardly any programming languages which take into account the established tools of linguistics; Only Larry Wall‘s Perl comes to mind, mostly because he’s a professional Linguist, too. I think many, if not most programming languages rely implicitly on the innate linguistic knowledge of its creator(s).

The Framework’s 12 Criteria

Note that the lexer’s code snippets are in .g4 format since we use ANTLR 4.

1. Alignment with Existing Language

Yes, and we’ll later discuss one feature which I’ve never seen in a programming language: keywords agreeing with the gender of variable names.

2. (Non-English) Keywords

Yes, and I use tools from Corpus Linguistics to prove the keywords’ semantic fit. This is especially important because…tada, I’m a second language speaker of Arabic (just an L2) and not a native speaker (L1). Nevertheless, I think that linguistically proving a keyword’s fit for purpose could prove universally beneficial.

3. (Non-Latin) Variable Names

Yes, of course.

// Start with an Arabic-script letter, then allow letters, diacritic marks,
// digits, _, and tatweel which is \u0640.
IDENTIFIER:
  ARABIC_LETTER (
    ARABIC_LETTER
    | ARABIC_MARK
    | JOIN_CONTROL
    | ID_DIGIT
    | '_'
    | '\u0640'
  )*;

4. (Non-English) Productions

No. This is a work in progress. I have not yet recognized a situation specific to Arabic where syntax productions must change. I believe that future HCI research within a larger user base will almost certainly raise some.

5. (Non-English) Numerals and Numbers

Yes, in fact it supports both the Eastern and Western versions of Arabic numerals. The lexer also accepts Latin numerals since the Unicode Consortium’s research shows these are increasing in observed usage throughout the Arab world. Remember, ArabicBASIC must conform to observed reality:

// Western + Arabic-Indic + Eastern Arabic-Indic digits.
fragment ID_DIGIT: [0-9\u0660-\u0669\u06F0-\u06F9];

6. Characters ‘without Meaning’

Yes, including the aforementioned tatweel and also the nearly infamous Zero-width joiner and its opposite the Zero-width Non-joiner:

// Join controls (not whitespace): ZWNJ, ZWJ.fragment JOIN_CONTROL: [\u200C\u200D]; 

I also consider whitespace to be part of this criterion:

// Unicode-aware whitespace skipping (excluding line terminators handled by EOL).
[\u0009\u000B\u000C\u0020\u00A0\u1680\u2000-\u200A\u202F\u205F\u3000]+ -> skip;

7. Diacritics

Incredibly, Yes; This was added recently and merits a future discussion in detail all on its own since they are optional because Arabic orthography also considers them optional in almost all cases (ditto for other Semitic languages except for Maltese given that it’s written in Latin letters):

fragment ARABIC_MARK:
[\u064B-\u065F\u0670\u06D6-\u06ED\u08D3-\u08E1\u08E3-\u08FF];

8. Alternative Keywords

Also, Yes. This will become very important later when we discuss diglossia in Arabic, which means that the spoken Arabic languages are quite different from the standard written and cross-national formal spoken varieties called Modern Standard Arabic (MSA), let alone the Classical Arabic of the Qur’an and other religious contexts like some Catholic parishes. Swidan and Hermans also consider gender agreement with variables in this criterion, too which I’ve addressed above.

9. Localized Punctuation

Yes, ArabicBASIC supports the following given how much they differ from their Latin counterparts (Latin semi-colon and question marks are not accepted even though Latin commas are; they just look weird in Arabic text):

ARABIC_SEMICOLON: '\u061B';
// Arabic and ASCII commas are both accepted for argument/identifier lists.
COMMA: [,\u060C]; 
fragment QMARK_AR: '؟';

I’m also considering various permutations of brackets and parentheses, but I still need to do my research with the CLDR group on this.

10. Right-to-Left Support

Well, absolutely. However, I will discuss in a later posting just how hard it is to mix bidirectional texts in a Terminal emulator…it’s quite painful and in many terminal implementations, simply does not work. Sometimes it’s just buggy and other times, the terminal’s docs state explicitly that it doesn’t support it, too bad.

11. Multilingual Programming

No. I don’t plan to, either. Swidan and Hermans present this criterion in the context of bilingual speakers; I’m half persuaded by this alone, but for ArabicBASIC I think it would defeat the purpose although the grammar does currently allow Latin variable names. This is only because it’s an artifact of the initial phases of coding the interpreter and not a conscious design choice. I haven’t decided if I’ll clean it up later.

12. Error Messages

Yes.

I think ArabicBASIC does pretty well at this stage and complies with most of the 12 criteria. Certainly, there’s more to do and I’m excited to push the boundaries even farther on just how Arabic can a programming language can get.

Sweden, A., Hermans, F. (2023). A Framework for the Localization of Programming Languages. Proceedings of the 2023 ACM SIGPLAN International Symposium on SPLASH-E, (SPLASH-E 2023), 13-25. https://doi.org/10.1145/3622780.3623645