FROM DATA TO KNOWLEDGE ANALYTICS: CAPABILITIES AND LIMITATIONS

Abstract

Data analytics is playing a central role in deriving useful information from large amounts of data available online in a variety of domains and applications. Analytics employs a wide array of methods ranging from classical statistical techniques to those exploiting the visual and cognitive capabilities of human users. In spite of all its capabilities, analytics at present seems to suffer from significant limitations in dealing with unstructured data and knowledge. This article explores the limitations and defines key requirements to be met by future developments in analytics. The article concludes with a sketch of true knowledge analytics which is capable of delivering insights from knowledge structures, not just tabular data.

Headnote

Data analytics is playing a central role in deriving useful information from large amounts of data available online in a variety of domains and applications. Analytics employs a wide array of methods ranging from classical statistical techniques to those exploiting the visual and cognitive capabilities of human users. In spite of all its capabilities, analytics at present seems to suffer from significant limitations in dealing with unstructured data and knowledge. This article explores the limitations and defines key requirements to be met by future developments in analytics. The article concludes with a sketch of true knowledge analytics which is capable of delivering insights from knowledge structures, not just tabular data.

Keywords: Analytics, Unstructured data, Knowledge, Capabilities, Limitations

1 INTRODUCTION

One of the key benefits of computerization comes from ready access to data. Extracting useful information from data has been a challenge for the field of information science. In the early stages of computerized data processing, data analysis was carried out in both scientific and business computing by applying well-known methods of statistics. Although many tools were developed, data analysis required expertise in both statistics and data processing. With the advent of the World Wide Web, social media and ubiquitous on-line access to data through personal computing devices and smart phones, data analytics is within the reach of everybody. Further advancements in programming and graphics technologies have made it possible to run analytical methods on current data and to generate colorful graphical renderings in real time. Open data initiatives of governments as well as non-governmental organizations have further democratized access to data by placing massive data repositories in the public domain. Ordinary citizens are beginning to look for useful trends, patterns and insightful guidance from such data sources.

The key question is whether analytics is ready to meet this challenge. Analytics thus far has focused mainly on well managed structured data, that is, collections of pieces of information in well defined formats found typically in spreadsheets or databases. Such well structured data can be readily classified into nominal, interval, ordinal and ratio data making data amenable to classical statistical analysis techniques. Unstructured data comprises of content in varied formats such as documents, images, emails, tweets, videos, blog posts and so on. Unstructured data is more valuable to organizations if we can extract actionable information from it which leads to knowledge. Knowledge itself is becoming accessible to machines increasingly in the form of semantic data represented in languages such as Resource Description Framework (RDF), Simple Knowledge Organization System (SKOS), Linked Open Data (LOD) and Web Ontology Language (OWL). Such representations include knowledge structures which are more complex than the simple tabular structure of data. They are typically organized in hierarchies or networks which are neither easily represented in tabular forms nor analyzed using classical statistical techniques. This new kind of open data poses further challenges by being a mix of structured and unstructured, numerical and textual, tabular and graph or network structured, authentic and questionable, clean and dirty, incomplete as well as redundant data. Are newer methods of analysis needed for handling such data? Are there such methods already developed in related areas of information science and technology such as natural language processing, Semantic Web, machine learning, and so on?

At present, analytics is being applied in various domains leading to many of its flavors such as Big Data Analytics, Visual Analytics, Graph Analytics, Social Network Analysis and so on. This article looks at how analytics has progressed as a discipline with its various capabilities and limitations. In particular, it considers the requirements of performing analytics on unstructured, semantic data or knowledge to raise questions about whether a different set of methodologies and algorithms is needed. It concludes with a sketch of how the newer methods can develop into true Knowledge Analytics, that is, analytics of knowledge structures in the near future.

2 ROLE OF ANALYTICS

With the rapid growth of online data, businesses are embracing analytics for discovering and reporting on values and outcomes. While analytics has shown promise in a wide range of verticals and applications, it is important to examine its capabilities and limitations especially in the context of unstructured data and knowledge, not merely conventional structured data. Current analytics applications focus primarily on structured business or governance data and use a variety of algorithms ranging from statistical quantitative techniques and numerical linear algebra methods to elementary string and text processing functions.

In this and the following section, we survey various types of analytics and their applications in different domains. We later analyze the future requirements of applying analytics to unstructured data and knowledge and conclude with a proposed framework with newer methods of analytics to meet the requirements of the future.

2.1 Analytics as Competitive Advantage

Many businesses have made analytics one of their key competitive strategies. The ability of analytics to discover new information and enable quantitative decision making in day-to-day as well as strategic initiatives is being seen as a competitive differentiator [1]. Questions being addressed through analytics include: "How do competitors obtain timely, sharp insights from the available information [2]?" and "Are competitors interpreting the signals from global economy in the right sense?". Making Analytics a top priority in their business processes enables organizations to know the what, why, and how of every consequence of business decisions in addition to getting a clear cut view of future trends and opportunities in business. MIT Sloan Management Review in association with the IBM Institute for Business Value has conducted research to understand the challenges and opportunities of business analytics by capturing insights from over 30 organizations of various sizes as well as individuals across 108 countries. Top performers consistently indicated that analytics is the key differentiator in business. Managing risk in business is counter-measured by the creation and analysis of proprietary information which helps detect the future of customer priorities [3].

Since analytics ensures greater accuracy in decision making processes by removing the reliance on "gut instinct", it is imperative in organizations to measure and track business results across time, geography and business verticals. Data is creating leads and contexts in which analytics wrings every last drop of value bringing insights for business opportunities [1].

2.2 Challenges in Analytics

According to a survey conducted by eHealth Initiative and the College of Healthcare Information Management Executives (CHIME), analytics is still in early stages of maturity. Descriptive analytics is employed more frequently compared to prescriptive and predictive analytics. While most organizations recognize the importance of analytics, the majority have not yet committed to investing in analytics and integrating it into their business and decision making processes.

Irish Data Analytics Landscape Survey conducted in December 2014 has highlighted challenges faced by data analytics projects. According to this survey, lack of skilled data analysts is the most difficult challenge (45%) followed by insufficient amounts of relevant data (35%), cost (25%), difficulty accessing suitable tools (23%), lack of corporate sponsorship (22%), data protection issues (11%) and lack of compelling business case (5%) [4].

Challenges which arise from technological issues while applying analytics in core business processes include modifying existing systems that run transactions and produce transactional data, collecting sufficient mass of data for reliable forecasting, providing extensive training to data analysts to recognize the flaws in data (e.g., missing data), and managing duplicate data and poor quality data [1].

Various numerical and algorithmic approaches being used to automatically analyze data fall short of certain features such as algorithm scalability, increasing data dimensions, and data heterogeneity. User assistance is still needed to iteratively refine the methods and interpret the findings in an intuitive manner [5]. Analytics is yet to systematically include quantitative measures of unstructured data such as views or comments on a product as Social ROI (Return On Investment) along with conventional business and financial parameters. Analytics is also immature when dealing with subjective and less tangible dimensions such as measuring performance of humans. Further, with more and more devices going mobile, it is a daunting task to perform analytics on mobile devices due to their smaller computing capabilities and user interfaces.

3 TYPES OF ANALYTICS

3.1 Core Analytics

Analytics is mostly applied to business data for gaining insights on opportunities and challenges in business through the use of extensive statistical methods. Businesses want to know what features in a product will give it more demand. A company may wish to forecast the total demand based on past demand combined with past and current economic conditions. Manufacturers of consumer electronics may need to understand the sentiments of their customers communicating over social media about their products. Some of the key analytics tasks performed to address such questions are: Hypothesis Testing, Profiling, Link Predictions, Associations, and Class Probability Estimation. These methods are also typically classified into those based on supervised and unsupervised learning or into dependency and interdependency analyses.

3.2 Big Data Analytics

Various dimensions and challenges of Big Data have been characterized based on its v's - volume, velocity, variety, value and veracity. The so called 4th Paradigm of Science is promising to change the very angle of looking at data through exploration [6]. This is especially attractive in current situations where data is becoming available from different times, locations, perspectives, topics, cultures, resolutions, qualities, and so forth.

Quite apart from Big Data, Linked Data or Linked Open Data (LOD), a type of semantic data, is also growing in adaptation as an ideal testbed for Big Data where many variables are semantically linked and controlled. Linked Data can be easily accessed, stored, and retrieved using open source tools. However, analytics of Big Data as well as Linked Data is at best in its infancy at this time; robust and scalable frameworks are needed for Big Data analytics before large scale applications can become commonplace.

3.3 Visual Analytics

The appeal of Visual Analytics is derived from the well-known ability of the human mind in recognizing patterns of significance in a visual presentation. Visual analytics combines this human capability with the number crunching and data management abilities of computers through visual mapping, model-based analysis and user interactions to gain insight into complex problems. Akin to the information visualization mantra [7], the visual analysis mantra [8] highlights the combination of numerical/algorithmic data analysis and interactive visual interfaces.

According to [5], interaction is the critical glue which integrates analytics, visualization and the human analyst. The commonly used phrase is "human in the loop" to indicate the need for human consultation in visual analytics. Using the "human is the loop" which emphasizes the need to seamlessly fit the analyst's cognitive processes into analytics, [5] suggests applying the context of the visualization process to assist analytics.

3.4. Graph Analytics

Graph analytics refers to techniques which use graph models to solve complex problems in analytics involving non-linear structures in the data. Apart from standard graph-theoretic algorithms for finding optimal paths, graph analytics makes use of techniques for measuring various types of centrality and for identifying sub-graphs with a well-defined structure [9].

For example, GreenGrid is a graph analytics tool developed for a power grid's geography or a circuit board's electronics and uses weighted force directed graph layout to model the power grid's attractive and repulsive forces. GreenGrid was used to detect potential vulnerabilities. It was learned that there is a fine difference between visualizing data and visualizing the theory governing the data while analyzing power grids. [5] also reaffirms the goal of visual analytics to integrate visual and analytical methods for broader knowledge and deeper understanding of the data.

3.5 Social Network Analysis

Social Network Analysis and Social Media Analytics are two seemingly similar yet different terms. Social media such as Facebook, Twitter, blogs and wikis produce massive amounts of information with the potential to tap into "the wisdom of the crowds" [10]. Social Media Analytics consists of analytics tools for reporting, dashboarding, visualization and search on information sourced from social media. With social media tearing apart the boundaries between authorship and readership, a novel feature of social media lies in the interleaving of the processes of sharing and generating information. Analytics on social media helps extract and analyze important online dialogues to get useful insights on active contributors. On the other hand, Social Network Analysis is part of advanced analytics which mainly focuses on identifying relationships, connections, communities and their influence on individuals and groups [11]. Social Network Analysis came into forefront as many sociologists began building on RadcliffeBrown's concept of social structure [12]. As the main focus is on relationships in social networks, concepts of analytics applicable to social networks differ from traditional statistical analytics [13].

In similar work on collaboration network and bibliographic citation analysis, users can visualize nodes in a layout based on semantic substrates which are meaningful attributes of nodes. Users can see interesting and trending patterns in the network based on grouping of meaningful attributes.

3.6 Cognitive Analytics

According to Lucker [14], Cognitive Analytics differs from traditional analytics in that it is modeled on the way the human brain processes information, takes action and learns from their consequences. Cognitive Analytics benefits from the feedback given to the analytics ecosystem which enables machine learning to handle new challenges in analytics.

3.7 Content Analytics

Content Analytics enables one to extract structured information from unstructured text. This structured information can be used to track, organize, and search content. "Content analytics allows people to find what they're looking for, not what they're searching for", says Stephen Ludlow, director of enterprise product marketing at OpenText. Content Analytics deals with extracting the metadata from the text that helps the user to arrive at the right place in any search task. According to IBM, the majority of information is already unstructured. Content analytics involves polishing unstructured data by extracting keywords and language that better enable measuring the use and effectiveness of the content. Content Analytics thus enables queries and analysis of both structured and unstructured content. From the perspectives of Information Retrieval and Binary Classification, precision and recall [15, 16] are measures of quality and completeness of retrieved content respectively. Metrics such as precision and recall cannot be optimized for tasks such as document retrieval from the Web and Internet Search when user's objectives are not defined precisely. Sudhir Holla, SVP of retail at Ugam, a provider of managed analytics for retail, adds "Content Analytics is at the heart of marketing industry to help realize which brand gains effective traffic and how to manage the traffic" [17] wherein some of the use cases provided include:

i. Human language inputs for search and discovery;

ii. Personalization and recommendations;

iii. Content consumption based customer understanding;

iv. Understanding what topics are driving engagement in editorials; and

v. Understanding readers sentiments based on postings on social media.

3.8 Learning Analytics

Learning Analytics encompasses analyzing data collected during the learning process for predicting, advising people's learning and reporting settings required to optimize learning. Learning Analytics involves academic analytics and educational data mining [18]. Academic Analytics involves marrying large datasets to statistical analytic and prediction techniques. Educational Data Mining is exploring the variety of data collected to analyze students and their learning environments.

3.9 Text Analytics

The process of deriving quality information from unstructured text through the means of trends and patterns is Text Analytics. Natural language processing, named entity recognition and co-reference resolution are some of the subtasks involved in the analysis of texts. Taking a business and customer satisfaction perspective, text analytics helps organizations get insight of opinions, issues and opportunities which would otherwise be buried in the text. Analyzing customer feedback and survey reports which are essentially textual, helps expose the reasons behind the feedback thereby helping the business strategist to assimilate the entire customer experience journey [19].

3.10 Bibliometrics/Informetrics/Scientometrics/Webometrics/Altmetrics

Publication metadata about books and periodicals, along with their authorship and citation networks provides a rich set of opportunities for analytics. Analyzing the quantitative aspects of such data has led to the birth of the research disciplines of Bibliometrics, Informetrics, Scientometrics and Webometrics. Bibliometrics has been defined by Pritchard [20] as the application of mathematical and statistical methods to books and other communication media. Bibliometrics was used to assess the overall development and growth of modern civilization. Informetrics is a related field dealing with the measurement and mathematical modeling of all aspects of information including its storage and retrieval. The output of scientific and technological work appears in the public domain as publications in conferences and journals. The term Scientometrics has been coined by V. V. Nalimov in 1960s [21] and since then scientometrics aims to bring measuring techniques for the evaluation of advancement of science along with its impact and relevance to the society [22]. Scientometrics has been successful in understanding the productivity of scientists while also determining the patterns of coherence in scientific creativity [22]. The application of Informetrics methods on the World Wide Web is Webometrics which investigates the Web from a quantitative viewpoint [23]. As the Web is dynamic and continues to evolve, it requires methods of asynchronous measurement and analysis. "No one can read everything" is the punch line of the new discipline Altmetrics. As scholarly work has moved to the Web, there are more significant traces of impact on the Web in forms such as online reference managers (e.g., Mendeley), tweets, scholarly blogs and social and professional networks (e.g., ResearchGate). In addition, articles are supplemented by datasets, experimental designs and code, semantic publishing, self publishing like microblogging, comments and annotations [24].

4 RDF ANALYTICS

With the development of Semantic Web, data containing self-descriptive semantics is becoming increasingly available in the form of Resource Description Framework (RDF) graphs, triple stores and Linked Open Datasets. Although they can be treated as structured data to some extent, the regular methods of analytics are not capable of exploiting their semantics or even their complex structure in the form of graphs and hierarchies. A few solutions for performing effective analytics on semantic data have emerged in the area known as RDF Analytics.

One such solution is RDF Analytical Schema proposed in [25] to analyze large semantic graphs. An analytical schema models the chosen dimensions and attributes for querying and analysis. Such schema instances enable analysis over a dynamic set of dimensions and measures in RDF data. Analytical query or a cube analyses facts (i.e., RDF triples) based on some dimensions for a particular attribute or measure. Various OLAP operations such as slice-dice, roll up and drill down have been used as cases for performance evaluation on analytical queries. Although RDF Analytical Schema is a good beginning for analyzing RDF multidimensional graph data, it relies on traditional non-semantic methods of analytics and falls short of taking advantage of true semantic data.

5 LIMITATIONS OF ANALYTICS

As seen by the wide range of methods in analytics as well as the variety of application areas, analytics is progressing to enable leaps in data driven discovery and decision making. However, one of the major unsolved issues is systematic handling of unstructured data. It is found that 80% of business related data is in unstructured form, especially text [26]. The majority of the methods of analytics work very well on structured data. Unstructured data requires an altogether different approach to analytics of such messy data. Although there are discovery systems including Verity's K2, Stratify's Discovery System and Inxight's SmartDiscovery which help explore unstructured data [27], there is a need to devise a broad framework for analyzing unstructured data.

Since a basic requirement in managing unstructured data is to make it searchable, content management systems have focused on content intelligence services such as search, classification and discovery. There is also suites of tools for natural language processing such as the General Architecture for Text Engineering which provide basic information extraction capabilities [28]. Perhaps the only known industry standard for content analytics is Unstructured Information Management Architecture (UIMA) which is prominently used in logistics analysis software systems. According to the specification of the UIMA Standard, assigning semantics to unstructured data makes it partly structured and the simplest way of assigning semantics is either wrapping regions of text with xml tags or extracting elements of a document and creating instances in a knowledge base [29]. To get the most out of unstructured data, we need to begin defining, designing and implementing analytical frameworks and operations on large masses of unstructured data.

Apart from textual unstructured data, semantic data and knowledge structures based on standards and recommendations of the World Wide Web Consortium's Semantic Web such as Resource Description Framework (RDF) data, triples, Linked Open Data and Web Ontology Language (OWL) ontology data are becoming increasingly available [30]. We can expect them to take greater prominence in the inputs to analytics in the future as they constitute the underpinning of open data and open science initiatives as well as scholarly work on machine learning and intelligence [31]. Semantic Web applications are also expected to generate diverse machine-understandable semantic content tagged with a variety of semi-standard vocabularies and ontological concepts. Novel methods and algorithms for analyzing such knowledge structures are needed to take analytics to a new era.

6 REQUIREMENTS OF SEMANTIC DATA

We propose that future methods of analytics capable of dealing with both unstructured and semantic data must meet the following requirements:

1. Managing non-tabular data, that is, data whose structure is more complex than that of spreadsheets or relational tables. Such data typically has elements of tabular as well as hierarchical (or tree-structured) and graph (or network) organization;

2. Quantitative analysis of graph-structured data, beyond those available in graph analytics, such as aggregation of measures in graph (or network) structures. For example, centrality measures may need to be computed at various levels of networks and sub-networks;

3. Quantitative analysis of hierarchical data, especially when such data is superimposed on graph structures. For example, attribute values may need to be aggregated at parent nodes in a hierarchy and also simultaneously propagated to super-networks from a sub-graph or sub-network;

4. Ability to aggregate information considering term equivalence as defined in external sources such as thesauri and ontologies;

5. Ability to integrate the structure of data with the structure of natural language sentences, texts, dialogues and discourses;

6. Ability to process multi-lingual data;

7. Managing multiple versions of semantic and ontological data which may be inconsistent with each other;

8. Managing multiple snapshots of time-varying data of the above kinds;

9. Managing authenticity and provenance of information sources;

10. Computationally efficient ways of making valid inferences from semantic data;

11. Computationally efficient techniques for validating data against both previous data and prescribed semantic constraints on the data; and

12. Enabling visualization of the above kinds of data.

The above mentioned requirements define the future shape of analytics for structured as well as unstructured semantic data which can be in various forms such as graphs, hierarchical networks, knowledge structures and ontologies. A sketch of such a Knowledge Analytics framework capable of performing semantic, analytics operations on semantic data is shown in Table 1.

7 CONCLUSION AND FUTURE WORK

Analytics has its roots in statistics right from the 18th century. From its early days, many communities have contributed to the development of analytics leading to a range of perspectives and approaches such as Business Intelligence, Big Data Analytics, Graph Analytics, Social Network Analysis, Cognitive Analytics, Content Analytics, Learning Analytics, Text Analytics, Bibliometrics, Informetrics, Scientometrics, Webometrics and RDF Analytics. Of late, the widespread availability of on-line data in a variety of domains has propelled both interest and investment in analytics with its promise of providing factual information along with insights for deeper understanding of data. The methods and algorithms of analytics work seemingly well on structured data but not so well for unstructured data and knowledge structures. We have attempted to list the gaps in the form of a dozen requirements to be met by analytics of the future. We have also outlined a framework for Knowledge Analytics for effective analysis of knowledge structures. In our ongoing work, we are further defining and formalizing the proposed methods of knowledge analytics while also implementing them to demonstrate their application and effectiveness.

8 ACKNOWLEDGMENT

This work is supported in part by the World Bank/Government of India research grant under the TEQIP programme (subcomponent 1.2.1) to the Centre for Knowledge Analytics and Ontological Engineering (KAnOE) at PES University, Bangalore, India.

References

REFERENCES

1. Davenport, Thomas H. and Jeanne G. Harris. (2007). Competing on Analytics: The New Science of Winning. Boston, Mass: Harvard Business School Press.

2. Steve Lavalle, et al, (2011). Big Data, Analytics and The Path from Insights to Value. MIT Sloan Management Review Rep., 52(2).

3. Adrian Slywotzky and John Drzik. (2005). Managing Uncertainty: Countering the Biggest Risk of All. Harward Business Review Rep.

4. The Analytics Store. (2014). Irish Data Analytics Landscape Survey 2014-2015. The Analytics Store Rep.

5. Endert, Alex, et al. (2014). The human is the loop: new directions for visual analytics." Journal of intelligent information systems. 43(3): 411-435.

6. Hitzler, Pascal, and Krzysztof Janowicz. (2013). Linked Data, Big Data, and the 4th Paradigm. Semantic Web. 4(3): 233-235.

7. Shneiderman, Ben. (1996). The eyes have it: A task by data type taxonomy for information visualizations. Visual Languages, Proceedings IEEE Symposium on. IEEE, 1996.

8. Keim, Daniel, et al. (2006). Challenges in visual data analysis. Information Visualization, 2006. IV 2006. Tenth International Conference on. IEEE, 2006.

9. Rajesh Bordawekar. (2015). Analysing Analytics, Mass: Morgan and ClayPool Publishers.

10. Leskovec, Jure. (2011). Social media analytics: tracking, modeling and predicting the flow of information through networks. Proceedings of the 20th international conference companion on World Wide Web. ACM.

11. James Kobielus. Social Media Analytics vs Social Network Analysis: Is there a real difference or Are you seeing double?, http://blogs.forrester.com/james_kobielus/10-07-07social_media_analytics_vs_social_network_analysis_there_real_difference_or_are_you_se eing_double

12. John Scott (2000). Social Network Analysis: A Handbook, Sage Publications.

13. Stanley Wasserman and Katherine Faust. (1994). Social Network Analysis: Methods and Applications, Cambridge University Press.

14. Scott Etkin, Analytics Trends 2015: Q and A with Deloitte's John Lucker. http://datainformed.com/analytics-trends-2015-q-and-a-with-john-lucker/

15. Buckland, Michael K. and Fredric C. Gey (1994). The relationship between recall and precision. JASIS, 45(1): 12-19.

16. Mahesh, Kavi. (1999). Text Retrieval Quality: A Primer. Oracle Technology Network (Technet.Oracle.com), Oracle Corporation Redwood Shores, California, USA, 1999, http://www.oracle.com/technetwork/testcontent/imt-quality-092464.html

17. Tamara Franklin, (2015). The State of Content Analytics 2015. Econtent Mag.

18. Siemens, George, and Ryan SJ d Baker. (2012). Learning analytics and educational data mining: towards communication and collaboration. Proceedings of the 2nd international conference on learning analytics and knowledge. ACM, 2012.

19. Text Analytics, http://www.medallia.com/text-analytics/

20. Pritchard, Alan. (1969). Statistical bibliography or bibliometrics? Journal of documentation 25: 348-349.

21. Nalimov, V.V. and Mul'chenko, Z.M. (1989). Study of Science Development as an Information Process. Scientometrics, 15: 33-43.

22. Sengupta, I.N. (1992). Bibliometrics, informetrics, scientometrics and librametrics: an overview. Libri, 42(2): 75-98.

23. Almind, Tomas C. and Peter Ingwersen. (1997). Informetric analyses on the World Wide Web: Methodological approaches to 'webometrics'. Journal of documentation 53(4): 404426.

24. Priem, J.; Taraborelli, D.; Groth, P. and Neylon, C. (2010). Altmetrics: A manifesto, 26. http://altmetrics.org/manifesto

25. Colazzo, Dario, et al. (2014). RDF analytics: lenses over semantic graphs. Proceedings Of The 23rd International Conference On World Wide Web. ACM.

26. Seth Grimes, Breakthrough Analysis, http://breakthroughanalysis.com/2008/08/01/ unstructured-data-and-the-80-percent-rule/

27. Blumberg, R. and Atre, S. (2003). The problem with unstructured data. DM Review, 13: 42-49.

28. Cunningham, Hamish. (2002). GATE, a general architecture for text engineering. Computers and the Humanities, 36(2): 223-254.

29. Ferrucci, David, and Adam Lally. (2004). UIMA: an architectural approach to unstructured information processing in the corporate research environment. Natural Language Engineering, 10(3-4): 327-348.

30. Linked Data, http://www.w3.org/standards/semanticweb/data

31. Soergel, Dagobert. (1999). The rise of ontologies or the reinvention of classification. JASIS 50(12): 1119-1120.