LandQv2: A MapReduce-Based System for Processing Arable Land Quality Big Data
| Autor: | Ahmed Eldawy, Zuliang Zhao, Guoqing Li, Xiaochuang Yao, Sijing Ye, Louai Alarabi, Mohamed F. Mokbel, Long Zhao, Dehai Zhu |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2018 |
| Předmět: |
Geographic information system
010504 meteorology & atmospheric sciences Range query (data structures) parallel processing Computer science Geography Planning and Development Big data 0211 other engineering and technologies spatial big data lcsh:G1-922 Shapefile 02 engineering and technology computer.software_genre 01 natural sciences Physical Geography and Environmental Geoscience Earth and Planetary Sciences (miscellaneous) MapReduce Computers in Earth Sciences Spatial analysis 021101 geological & geomatics engineering 0105 earth and related environmental sciences Database Application programming interface business.industry computer.file_format File format GIS Geomatic Engineering Networking and Information Technology R&D (NITRD) arable land quality arable land quality (ALQ) Data pre-processing business computer lcsh:Geography (General) |
| Zdroj: | ISPRS International Journal of Geo-Information Volume 7 Issue 7 ISPRS International Journal of Geo-Information, vol 7, iss 7 ISPRS International Journal of Geo-Information, Vol 7, Iss 7, p 271 (2018) |
| ISSN: | 2220-9964 |
| DOI: | 10.3390/ijgi7070271 |
| Popis: | Arable land quality (ALQ) data are a foundational resource for national food security. With the rapid development of spatial information technologies, the annual acquisition and update of ALQ data covering the country have become more accurate and faster. ALQ data are mainly vector-based spatial big data in the ESRI (Environmental Systems Research Institute) shapefile format. Although the shapefile is the most common GIS vector data format, unfortunately, the usage of ALQ data is very constrained due to its massive size and the limited capabilities of traditional applications. To tackle the above issues, this paper introduces LandQv2, which is a MapReduce-based parallel processing system for ALQ big data. The core content of LandQv2 is composed of four key technologies including data preprocessing, the distributed R-tree index, the spatial range query, and the map tile pyramid model-based visualization. According to the functions in LandQv2, firstly, ALQ big data are transformed by a MapReduce-based parallel algorithm from the ESRI Shapefile format to the GeoCSV file format in HDFS (Hadoop Distributed File System), and then, the spatial coding-based partition and R-tree index are executed for the spatial range query operation. In addition, the visualization of ALQ big data with a GIS (Geographic Information System) web API (Application Programming Interface) uses the MapReduce program to generate a single image or pyramid tiles for big data display. Finally, a set of experiments running on a live system deployed on a cluster of machines shows the efficiency and scalability of the proposed system. All of these functions supported by LandQv2 are integrated into SpatialHadoop, and it is also able to efficiently support any other distributed spatial big data systems. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|