(This is the first blog post in a longer series of posts about the sources of OpenITI.)
Al-Maktaba al-Shamila (‘The comprehensive library’, often referred to simply as Shamela) is a free software that aims at providing a digital research environment for Islamic scholars and comes with a large collection of Arabic primary sources and secondary texts. It has millions of users in the Islamic world and is also widely used in Western academia. It is currently the largest source of texts in the OpenITI corpus: about 45% of the texts in the corpus come from Shamela.
In its first iteration, it was called al-Mawsuʿa al-Shamila (‘The comprehensive encyclopaedia’). The program was created by an Egyptian member of the Ahl al-hadith forum known only as Nafiʿ (often referred to affectionately on specialised internet fora as al-akh Nafiʿ, ‘Brother Nafiʿ’, or duktur/al-ustadh Nafiʿ, ‘Dr/Prof. Nāfiʿ’). When the program, containing 969 free books, first appeared on internet fora for Islamic scholars in early 2005, it met with a lot of enthusiasm – so much so that the pioneering publishing house of digital texts, Markaz al-Turath li-l-Baramijat (the publisher of al-Jamiʿ al-Kabir, the other main source of texts in our corpus), launched an acrimonious attack against the program on the Ahl al-Hadith forum, accusing Nafiʿ of stealing its software and content. This post led to such a heated discussion that the forum moderators had to remove the thread and ban all content related to al-Mawsuʿa al-Shamila on 4 May 2005. A new discussion thread was eventually opened on 7 May, with a statement from al-Turath and a response from the forum moderators. In its statement, al-Turath apologised for its earlier reaction before explaining in detail why it considered the program an act of theft (of digitised texts and specialised data) that would kill off not only the company but also all industry-led digitisation efforts. In their reaction, the forum moderators thanked al-Turath for its huge service to the field and Islam but also defended its members involved in the creation and distribution of the program as driven only by love for Islam. The discussion lasted for weeks and eventually prompted Nafiʿ, the creator of the program himself, to react in a post on the forum. In his statement, he claimed he had developed the program for himself, to enable him to simultaneously search the different collections of digital books, all of which he had legally. While admitting these included books from al-Turath, he strenuously denied that Shamela was ‘al-Turath in another jacket’, adducing proof that he had included books and data from many different sources. He claimed not to have shared his program with more than a handful of scholars and to have been surprised when he noticed, two years later, another member of the Ahl al-Hadith forum, with whom he had not shared the program and whom he did not even know, offering it (for free) to other members. He also vowed to create a new version of the program that did not include any book taken from a commercial product. After seemingly conciliatory rhetoric from both sides, the discussion turned bitter again, resulting in repeated accusations of piracy and calls for a boycott flying in both directions.
In January 2006 Nafiʿ finally published the promised second version (al-isdar al-thani) of the program, under the title al-Maktaba al-Shamila. As he had vowed, it did not include material taken (directly) from al-Turath products, only material from free websites and internet fora. This meant that it could not offer a number of important features of the first version: the page numbers of most texts did not agree with any published edition, and services such as the identification of Hadith transmitters and historical figures, which had been based on databases from al-Turath, were absent. On the flip side, the program was still free and contained a much larger number of texts than the first version, ca. 1,800 books.
In addition to a greatly increased collection of digitised books, the most important feature of the second version was that it was the first program of its kind that offered users the possibility to import their own texts into the program so they could create their own personal libraries. At the same time, this opened the door for people to keep using the program for reading and searching texts from companies such as al-Turath, without Shamela having to take responsibility for this. Shamela was the first program of its kind to offer this kind of feature, and the feature probably played an important role in the program’s great success.
Using this feature, a number of groups and individuals started publishing their own versions of al-Maktaba al-Shamila with many additional texts (e.g. the augmented Shamela collection of Islamport.com contained 3,300 books already in September 2006 and more than 5,000 books in March 2007; al-Maktaba al-Shamila al-Dhahabiyya contained 29,000 books in 2019). A number of these apocryphal shamela collections were created by groups from minority (non-Sunni) Islamic communities (e.g., al-Maktaba al-Zaydiyya al-Shamila, al-Maktaba al-Shamila al-Ibadiyya) whose books were not represented in the main Shamela collection. We will discuss these unofficial shamela collections in a later blog.
At first, Shamela Version 2 was distributed via internet fora, as the first version had been, due to limited stable internet access and bandwidth in much of the world at that time. The book collection was downloadable from a variety of internet fora and file sharing websites (including the then very recent website waqfeya.net), and users were encouraged to share copies of the program on CDs with people who did not have stable internet access.
The third version, released on 3 June 2008, represented an audacious step: Shamela decided to create a dedicated collection of books of its own that recreated the features lost in the second version after the dispute with al-Turath. Books from this official Shamela collection would have page numbers that agreed with the printed editions, and the program would provide special features such as a takhrij function for Hadith works (which would identify the sources for a Hadith). However, users would still be able to use the program for both the official collection and external books (using an improved import mechanism). An additional well-received feature was that both official and external books could be linked to scanned pages from the original editions. The official book collection available on the website initially contained only a handful of books: since Shamela could not base its collection on the digitising efforts of al-Turath and other companies, it had to invest a lot of effort in annotating the books in the official collection. The old books from Version 2 remained available, but from a separate page. It was not until November 2010 that the first official collection of Shamela books was launched. It contained 5,300 books, almost double the number of books on the old website at the time (2,861 books). Simultaneously, a separate web repository was launched where books provided by volunteers, which were not part of the official release, could be stored. This users’ repository currently contains almost 5,500 books.
However, already in February 2006, an unidentified entity registered the domain name Shamela.ws (.ws is the internet country code for West Samoa, but it was popular in the early 2000s because it could be seen as an abbreviation of ‘website’ or ‘web service’ rather than referring to a specific country). Shamela.ws became the official website for al-Maktaba al-Shamila in early 2007 and remains so today, and the software and the book collection (in its entirety or as separate books) can be downloaded from it.
In those early years, al-Maktaba al-Shamila was run by a group of volunteers. Their colossal accomplishment did not go unnoticed: Arab and Islamic governments and institutions came under criticism because a group of volunteers had accomplished what they had not been able to do with their multi-billion budgets.
In 2010, the company Sharikat al-maktaba al-shamila li-l-barmajiyyat was founded (by Nafiʿ?) to take sole responsibility for al-Maktaba al-Shamila and to coordinate everything connected to it. The company is based in Egypt, and so is the team responsible for the preparation of the books. Until 2012, it was not supported by any institution and depended only on charitable gifts from individuals.
In 2012, al-Maktaba al-Shamila came under the patronage/sponsorship (riʿaya) of a Saudi institution, the Daʿwa Center, in the al-Rawda neighbourhood of the Saudi capital Riyadh (al-Maktab al-taʿawuni li-l-daʿwa wa-l-irshad wa-tawʿiyyat al-jaliyat bi-hayy al-Rawda, ‘the cooperative office for propagation and guidance for expatriates in al-Rawḍa district’). Such offices, aimed at spreading Islam, especially among the huge immigrant communities in the kingdom, are present in many neighbourhoods in Saudi Arabia. The first effect of this patronage seems to have been a massive influx of funding, allowing a rapid increase in manpower (literally; all employees listed in this period were men). A pamphlet on al-Maktaba al-Shamila, published by al-Rawda in 2013 (?), publicises the growth of the team behind Shamela following the start of its patronage: from one programmer to four (still based in Egypt), and from five data entry employees to thirty. The additional funding was invested in creating new versions of the program to run on iPhone (released in September 2012) and Android (released in September 2013) devices, digitising and correcting new books, and developing additional features for users. The increase in funding was accompanied by a professionalisation of the communication department: official Shamela pages were created on Twitter and Facebook, and these started publishing news on the library from June 2012 onwards.
The current status of al-Rawda’s patronage of Shamela is unclear: a banner proclaiming the support of al-Rawda for al-Maktaba al-Shamila was displayed on the shamela.ws website from May/June 2012 until October 2015, when it was replaced by a banner from another Saudi organisation, Awqaf al-Rajihi (a private charity set up by the self-made Saudi businessman Salih bin ʿAbd al-ʿAziz al-Rajihi, who died in 2011), until August 2017. The Rawda banner reappeared for a short while (between November 2017 and May 2018) but has since then disappeared, without replacement.
I have not found direct references to the end of the relationship, but since March 2020 the link to al-Maktaba al-Shamila that featured prominently at the top of al-Rawda’s web page has also disappeared.
In March 2020, the fourth version of the software was launched, with a complete overhaul of the database system behind the program (sqlite instead of MS Access) and a new search engine (ElasticSearch). Shamela seemed to have reinvented itself and was well on its way to reaffirming its dominant status among Arabic digital text corpora. However, in the last couple of months, all of Shamela’s official communication channels (Facebook and shamela.ws since May, Twitter since July) have gone silent.
NOTE: this short history of Shamela was gathered mostly from the archives of the website shamela.ws (long live the Internet Archive!) and the Ahl al-Hadith forum (partly archived in Shamela). I’d be grateful if readers could point out other sources.
Shamela timeline:
-
2003 (?): program created by Nafiʿ, with 969 books
-
April 2005: program distributed for free on the Ahl al-Hadith forum, under the name al-Mawsuʿa al-Shamila
-
May 2005: Complaint by al-Turath; Ahl al-Hadith bans distribution of the program
-
January 2006: Version 2 (al-Maktaba al-Shamila al-isdar al-thani) launched, with ca. 1,800 books
-
March 2006: Islamport.com launched as a ‘search engine with a huge electronic scientific library’ with 1,800 books from al-Maktaba al-Shamila and 100 from other sources
-
February 2006: registration of website shamela.ws (will not become active until early 2007)
-
March 2007: Shamela.ws becomes the official website of al-Maktaba al-Shamila, coinciding with an update of the program to version 2.11
-
June 2008: Shamela Version 3 launched, with a small collection of books with the official Shamela stamp
-
November 2010: first release of the official collection of Shamela books (5,300 books) and of a separate web repository where books uploaded by members could be stored
-
March 2012: launch of the online browsable version of the official collection
-
June 2012: second release of the official collection of Shamela books (6,111 books)
-
September 2012: launch of Shamela for iPhone and iPad
-
September 2013: launch of Shamela for Android
-
October 2015–August 2017: Shamela sponsored by Awqaf al-Rajihi
-
December 2015: technical development of version 3 halted (v. 3.64)
-
August 2017: launch of al-maktaba.org, which presents itself as ‘the modern al-Maktaba al-Shamila’ (but is unrelated) and offers an updated browse and search interface to al-Maktaba al-Shamila
-
May 2019: last books added to the official collection of al-Maktaba al-Shamila
-
April 2020: Shamela version 4 launched, with ca. 7,000 books