Abstract: The heterogeneity and the lack of structure of World Wide Web make automated discovery, organization, and management of Web-based information a non-trivial task. Traditional search and indexing tools provide some comfort to users, but they genera

MANAGING WEB-BASED INFORMATION

Marco Scotto, Alberto Sillitti, Tullio Vernazza

DIST – Università di Genova,Via Opera Pia, 13, I-16145 Genova, Italy

Email: {scotto, alberto, tullio}@dist.unige.it

Giancarlo Succi

Libera Università di Bolzano, Piazza Domenicani, 3, I-39100 Bolzano, Italy

Email: Giancarlo.Succi@unibz.it

Keywords: Abstract:

Web Mining, Information Retrieval

The heterogeneity and the lack of structure of World Wide Web make automated discovery, organization, and management of Web-based information a non-trivial task. Traditional search and indexing tools provide some comfort to users, but they generally provide neither structured information nor categorize, filter, or interpret documents in an automated way. In recent years, these factors have prompted the need for

developing data mining techniques applied to the web, giving rise to the term “Web Mining”. This paper introduces the problem of web data extraction and gives a brief analysis of the various techniques to address it. Then, News Miner, a tool for Web Content Mining applied to the news retrieval is presented.

1 INTRODUCTION

The World Wide Web has become a huge source of information, but its content cannot be manipulated in a general way due to two main issues:

Finding relevant information is a difficult

task because the web is unstructured. Search engines, such as Altavista, Google, Lycos and many others, provide some comfort to users, but their query facilities are often limited and the results come as HTML pages

Most of the information, present on the

web, is stored as HTML pages. HTML is a semi-structured format designed to describe the layout of web pages not their content, and machines hardly process HTML.

These factors have prompted the need for developing data mining techniques applied to the web, giving rise to the term “Web Mining”. This paper introduces the problem of the extraction of information from the web and analyzes the various techniques to approach it. Then, it presents News Miner, a tool for news extraction, integration and

presentation, based on Web Content Mining. News Miner is a server side application that periodically scans a set of news sites, integrates news building a repository, and makes them available to an application server that use such information to build user-specific web pages. In particular, it organizes HTML documents, which are semi-structured, into structured XML documents, using XSLT (eXtensible Stylesheet Language Transformation) and XPath (XML Path Language). This paper is organized as follows: section 2 introduces Web Mining; section 3 analyzes the different types of Web Mining; section 4 describes the design and the implementation of News Miner; finally, section 5 draws the conclusions.

2 WEB MINING

Web Mining can be broadly defined as “the discovery and analysis of useful information from the World Wide Web” (Madria et al., 1999). In Web Mining, data can be collected at different levels: server side, client side, proxy servers, or obtained from an organization’s database. For instance, data can be stored in browser caches or in cookies at client level, and in access log files at server or proxy level. Web Mining can be decomposed into four