oai:www.ldc.upenn.edu:LDC2012T09
| |
OLAC Record oai:www.ldc.upenn.edu:LDC2012T09 |
| Metadata | ||
| Title: | Arabic-Dialect/English Parallel Text | |
| Access Rights: | Licensing Instructions for Subscription & Standard Members, and Non-Members: http://www.ldc.upenn.edu/language-resources/data/obtaining | |
| Bibliographic Citation: | Raytheon BBN Technologies, Linguistic Data Consortium, and Sakhr Software. Arabic-Dialect/English Parallel Text LDC2012T09. Web Download. Philadelphia: Linguistic Data Consortium, 2012 | |
| Contributor: | Raytheon BBN Technologies | |
| Linguistic Data Consortium | ||
| Software, Sakhr | ||
| Date (W3CDTF): | 2012 | |
| Date Issued (W3CDTF): | 2012-06-15 | |
| Description: | *Introduction* Arabic-Dialect/English Parallel Text was developed by Raytheon BBN Technologies (BBN), LDC and Sakhr Software and contains approximately 3.5 million tokens of Arabic dialect sentences and their English translations. *Data* The data in this corpus consists of Arabic web text as follows: 1. Filtered automatically from large Arabic text corpora harvested from the web by LDC. The LDC corpora consisted largely of weblog and online user groups and amounted to around 350 million Arabic words. Documents that contained a large percentage of non-Arabic or Modern Standard Arabic (MSA) words were eliminated. A list of dialect words was manually selected by culling through the Levantine Fisher (LDC2005S07, LDC2005T03, LDC2007S02 and LDC2007T04) and Egyptian CALLHOME speech corpora (LDC97S45, LDC2002S37, LDC97T19 and LDC2002T38) distributed by LDC. That list was then used to retain documents that contained a certain number of matches. The resulting subset of the web corpora contained around four million words. Documents were automatically segmented into passages using formatting information from the raw data. 2. Manually harvested by Sakhr Software from Arabic dialect web sites. Dialect classification and sentence segmentation, as needed, and translation into English were performed by BBN through Amazons Mechanical Turk. Arabic annotators from Mechanical Turk classified filtered passages as being either MSA or one of four regional dialects: Egyptian, Levantine, Gulf/Iraqi or Maghrebi. An additional General dialect option was allowed for ambiguous passages. The classification was applied to whole passages rather than individual sentences. Only the passages labeled Levantine and Egyptian were further processed. The segmented Levantine and Egyptian sentences were then translated. Annotators were instructed to translate completely and accurately and to transliterate Arabic names. They were also provided with examples. All segments of a passage were presented in the same translation task to provide context. *Samples* Please follow this link for a sample of the data in this release. *Updates* None at this time. | |
| Extent: | Corpus size: 79039 KB | |
| Identifier: | LDC2012T09 | |
| https://catalog.ldc.upenn.edu/LDC2012T09 | ||
| ISBN: 1-58563-617-7 | ||
| ISLRN: 833-244-299-656-3 | ||
| DOI: 10.35111/1hqm-te15 | ||
| Language: | English | |
| Egyptian Arabic | ||
| North Levantine Arabic | ||
| South Levantine Arabic | ||
| Language (ISO639): | eng | |
| arz | ||
| apc | ||
| ajp | ||
| License: | LDC User Agreement for Non-Members: https://catalog.ldc.upenn.edu/license/ldc-non-members-agreement.pdf | |
| Medium: | Distribution: Web Download | |
| Publisher: | Linguistic Data Consortium | |
| Publisher (URI): | https://www.ldc.upenn.edu | |
| Relation (URI): | https://catalog.ldc.upenn.edu/docs/LDC2012T09 | |
| Rights Holder: | Portions © 2012 Raytheon BBN Technologies, © 2012 Sakhr Software, © 1997, 2002, 2005-2009, 2012 Trustees of the University of Pennsylvania | |
| Type (DCMI): | Text | |
| Type (OLAC): | primary_text | |
OLAC Info |
||
| Archive: | The LDC Corpus Catalog | |
| Description: | http://www.language-archives.org/archive/www.ldc.upenn.edu | |
| GetRecord: | OAI-PMH request for OLAC format | |
| GetRecord: | Pre-generated XML file | |
OAI Info |
||
| OaiIdentifier: | oai:www.ldc.upenn.edu:LDC2012T09 | |
| DateStamp: | 2020-11-30 | |
| GetRecord: | OAI-PMH request for simple DC format | |
Search Info | ||
| Citation: | Raytheon BBN Technologies; Linguistic Data Consortium; Software, Sakhr. 2012. Linguistic Data Consortium. | |
| Terms: | area_Africa area_Asia area_Europe country_EG country_GB country_JO country_SY dcmi_Text iso639_ajp iso639_apc iso639_arz iso639_eng olac_primary_text | |