| Autor: |
Geunseong Jung, Sungjae Han, Hansung Kim, Kwanguk Kim, Jaehyuk Cha |
| Jazyk: |
angličtina |
| Rok vydání: |
2022 |
| Předmět: |
|
| Zdroj: |
IEEE Access, Vol 10, Pp 129958-129969 (2022) |
| Druh dokumentu: |
article |
| ISSN: |
2169-3536 |
| DOI: |
10.1109/ACCESS.2022.3229080 |
| Popis: |
Extracting the main content from a web page is essential in various applications such as web crawlers and browser reader modes. Existing extraction methods using text-based algorithms and features for English text can be ineffective for non-English web pages. This study proposes a main content extraction method that obtains visual and structural features from the rendered web page. Our method uses the first impression area (FIA), a part of a web page that users initially view. In this area, websites have applied many techniques that enable users to find the main content easily. Using the non-textual properties in the FIA, our method selects three points with high content area density and expands the area from each point until it meets several structural and visual-based conditions. We evaluated our method, browsers’ (Mozilla Firefox and Google Chrome) reader modes, and existing main content extraction methods on multilingual datasets using two measures: Longest Common Subsequences and matched text blocks. The results showed that our method performed better than other methods in both English (up to 46%, matched text blocks $\mathrm {\mathbf {F_{0.5}}}$ ) and non-English (up to 42%, matched text blocks $\mathrm {\mathbf {F_{0.5}}}$ ) web pages. |
| Databáze: |
Directory of Open Access Journals |
| Externí odkaz: |
|
