(This is the first blog post in a longer series of posts about the sources of OpenITI)
Al-Maktaba al-Shāmila (“The comprehensive library”, often referred to simply as Shamela) is a free software that aims at providing a digital research environment for Islamic scholars, and comes with a large collection of Arabic primary sources and secondary texts. It has millions of users in the Islamic world, and is also widely used in academia. It is currently the largest source of texts in the OpenITI corpus: about 45% of the texts in the corpus come from Shamela.
In its first iteration, it was called al-mawsūʿa al-shāmila (“The Comprehensive Encyclopaedia”). The program was created by an Egyptian member of the Ahl al-Ḥadīth forum known only as Nāfiʿ (often referred to affectionately on specialized internet fora as al-akh Nāfiʿ “Brother Nāfiʿ” or duktūr / al-ustādh Nāfiʿ “Dr. Nāfiʿ”). When the program, containing 969 free books, first appeared on internet fora for Islamic scholars in early 2005, it met with a lot of enthusiasm. So much so that the pioneering publishing house of digital texts, Markaz al-Turāth lil-barāmijāt (the publisher of al-Jāmiʿ al-Kabīr, that other main source of texts in our corpus), launched an acrimonious attack against the program on the Ahl al-Ḥadīth forum, accusing Nāfiʿ of stealing its software and content. This post led to such a heated discussion that the forum moderators had to remove the thread, and ban all content related to al-mawsūʿa al-shāmila on 4 May 2005. A new discussion thread was eventually opened on 7 May, with a statement from al-Turāth and a response from the forum moderators. In its statement, al-Turāth apologized for their earlier reaction, before explaining in detail why it considered the program an act of theft (of digitized texts and specialized data) that would kill off not only the company but also all industry-led digitization efforts. In their reaction, the forum moderators thanked al-Turāth for their huge service to the field and Islām, but also defended its forum members involved in the creation and distribution of the program as driven only by love for Islam. The discussion lasted for weeks, and eventually prompted Nāfiʿ, the creator of the program himself, to react in a post on the forum. In his statement, he claimed he had developed the program for himself, to enable him to simultaneously search the different collections of digital books all of which he had legally. While admitting these included books from al-Turāth, he strenuously denied that Shamela was “al-Turāth in another jacket”, adducing proof that he had included books and data from many different sources. He claimed not to have shared his program with more than a handful of scholars, and to have been surprised when he noticed, two years later, another member of the Ahl al-Ḥadīth forum, with whom he had not shared the program, and whom he did not even know, offering it (for free) to other members. He also vowed to create a new version of the program that did not include any book taken from a commercial product. After seemingly conciliatory rhetoric from both sides, the discussion turned bitter again, resulting in repeated accusations of piracy and calls for a boycott flying in both directions.
In January 2006 Nāfiʿ finally published the promised second version (al-iṣdār al-thānī) of the program, under the title al-Maktaba al-Shāmila (“The Comprehensive Library”). As he had vowed, it did not include material taken (directly) from al-Turāth products, but only from free websites and internet fora. This meant that it could not offer a number of important features of the first version: the page numbers of most texts did not agree with any published edition, and services like identification of ḥadīth transmitters and historical figures, which had been based on databases from al-Turāth, were absent. On the flip side, the program was still free, and contained a much larger number of texts than the first version, ca. 1800 books.
In addition to a greatly increased collection of digitized books, the most important feature of the second version was that it was the first program of its kind that offered the user the possibility to import their own texts into the program so they could create their own personal library. At the same time, this opened the door for people to keep using the program for reading and searching texts from companies like al-Turāth, without Shamela having to take responsibility for this. Shamela was the first program of its kind to offer this kind of feature, and it probably played an important role in its great success.
Using this feature, a number of groups and individuals started publishing their own versions of al-Maktaba al-Shāmila with many additional texts (e.g., the augmented Shamela collection of Islamport.com contained already 3.300 books in September 2006 and more than 5.000 books in March 2007; al-Maktaba al-Shāmila al-Dhahabiyya contained 29.000 books in 2019). A number of these apocryphal shamela collections were created by groups from a minority (non-Sunnī) Islamic community (e.g., al-Maktaba al-Zaydiyya al-Shāmila, al-Maktaba al-Shāmila al-Ibāḍiyya), whose books were not represented in the main Shamela collection. We will discuss these unofficial shamela collections in a later blog.
At first, Shamela Version 2 was distributed via internet fora, as was the first version: due to limited stable internet access and bandwidth in much of the world at that time. The book collection was downloadable from a variety of internet fora and file sharing websites (including the then very recent website waqfeya.net); and users were encouraged to share copies of the program on CDs with people who did not have stable internet access.
The third version, released on 3 June 2008, represented an audacious step: Shamela decided to create a dedicated collection of books of its own that recreated the features lost in the second version after the dispute with al-Turāth. Books from this official Shamela collection would have page numbers that agreed with the printed editions, and would enable special features like a takhrīj function for ḥadīth works (which would identify the sources for a ḥadīth). However, the user would still be able to use the program for both the official collection and external books (using an improved import mechanism). An additional well-received feature was that both official and external books could be linked to scanned pages from the original editions. The official book collection available on the website initially contained only a handful of books: since they could not base themselves on the digitizing efforts of al-Turāth et al., this meant Shamela had to invest a lot of effort in annotating the books in the official collection. The old books from version 2 remained available, but from a separate page. It was not until November 2010 that the first official collection of Shamela books was launched. It contained 5300 books, almost double the number of books on the old website at the time (2861 books). Simultaneously, a separate web repository was launched where books provided by volunteers, and which were not part of the official release, could be stored. This users’ repository currently contains almost 5500 books.
However, already in February 2006, an unidentified entity registered the domain name Shamela.ws (.ws is the internet country code for West Samoa, but was popular in the early 2000s because it could be seen as an abbreviation for “Website” or “Webservice” rather than referring to a specific country). Shamela.ws became the official website for al-Maktaba al-Shāmila from early 2007 till today, from which the software and the book collection (in its entirety or as separate books) can be downloaded.
In those early years, al-Maktaba al-Shāmila was run by a group of volunteers. Their colossal accomplishment did not go unnoticed: Arab and Islamic governments and institutions came under criticism because a group of volunteers had accomplished what they had not been able to do with their multi-billion budgets.
In 2010, the company “Sharikat al-maktaba al-shāmila li-al-barmajiyyāt” was founded (by Nāfiʿ?) to take sole responsibility of al-Maktaba al-Shāmila and coordinate everything connected to it. The company is based in Egypt, and so is the team responsible for the preparation of the books. Until 2012, it was not supported by any institution, but depended only on charitable gifts from individuals.
In 2012, al-Maktaba al-Shāmila came under the patronage/sponsorship (riʿāya) of a Saudi institution, the Daʿwa center in the al-Rawḍa neighbourhood of the Saudi capital Riyad (al-maktab al-taʿāwunī li-al-daʿwa wa-al-irshād wa-tawʿiyyat al-jāliyāt bi-ḥayy al-Rawḍa, “the cooperative office for propagation and guidance for expatriates in al-Rawḍa district”). Such offices, aimed at spreading Islam, especially among the huge immigrant communities in the kingdom, are present in many neighbourhoods in Saudi Arabia. The first effect of this patronage seems to have been a massive influx of funding, allowing a rapid increase of the manpower (literally, all employees listed in this period were men). A pamphlet on al-Maktaba al-Shāmila, published by al-Rawḍa in 2013 (?), publicizes the growth of the team behind Shamela following the start of its patronage: from one programmer to four (still based in Egypt), and from five data entry employees to thirty. The additional funding was invested in creating new versions of the program to run on iPhone (released in September 2012) and Android (released in September 2013) devices, digitizing and correcting new books, and development of additional features for users. The increase in funding was accompanied by a professionalization of the communication department: official Shamela pages were created on Twitter and Facebook, which started publishing news on the library from June 2012 onwards.
The current status of al-Rawḍa’s patronage of Shamela is unclear: a banner proclaiming the support of al-Rawḍa for al-Maktaba al-Shāmila was displayed on the shamela.ws website from May / June 2012 until October 2015, when it was replaced by a banner from another Saudi organization, Awqāf al-Rājiḥī (a private charity set up by the Saudi self-made businessman Ṣāliḥ bin ʿAbd al-ʿAzīz al-Rājiḥī, who died in 2011) until August 2017. The Rawḍa banner re-appeared for a short while (between November 2017 and May 2018) but has since then disappeared, without replacement.
In March 2020, the fourth version of the software was launched, with a complete overhaul of the database system behind the program (sqlite instead of MS Access) and a new search engine (ElasticSearch). Shamela seemed to have re-invented itself, and well on its way to re-affirm its dominant status among Arabic digital text corpora. However, in the last couple of months, all of Shamela’s official communication channels (Facebook and shamela.ws since May, Twitter since July) have gone silent.
- 2003 (?): program created by Nāfiʿ, with 969 books
- April 2005: program distributed for free on Ahlalhdeeth forum, under the name al-Mawsūʿa al-Shāmila
- May 2005: Complaint by al-Turāth, Ahlalhdeeth bans distribution of the program
- January 2006: Version 2 (al-Maktaba al-Shāmila al-iṣdār al-thānī) launched, with ca. 1800 books
- March 2006: Islamport.com launched as a “search engine with a huge electronic scientific library” with 1800 books from al-Maktaba al-Shāmila and 100 from other sources
- February 2006: registration of website shamela.ws (will not become active until early 2007)
- March 2007: Shamela.ws becomes the official website of al-Maktaba al-Shāmila, coinciding with an update of the program to version 2.11.
- June 2008: Shamela Version 3 launched: with small collection of books with official Shamela stamp
- November 2010: First release of the official collection of Shamela books (5300 books), and of a separate web repository where books uploaded by members could be stored.
- March 2012: launch of the online browsable version of the official collection
- June 2012: Second release of the official collection of Shamela books (6111 books)
- September 2012: launch of Shamela for iPhone and iPad
- September 2013: launch of Shamela for Android
- October 2015 – August 2017: Shamela sponsored by Awqāf Muḥammad ʿAbd al-ʿAzīz al-Rajḥī
- December 2015: Technical development of version 3 halted (v. 3.64)
- August 2017: launch of al-maktaba.org, which presents itself as “the modern al-Maktaba al-Shāmila” (but is unrelated) and offers an updated browse and search interface to al-Maktaba al-Shāmila
- May 2019: last books added to the official collection of al-Maktaba al-Shamela
- April 2020: Shamela version 4 launched, with ca. 7000 books.