Big data enables generation of intelligence in two somewhat complementary ways: The first is to use big data as training data for supervised machine learning, which would enable development of autonomous intelligent systems that can potentially replace humans in performing various tasks as in the case of self-driving cars or chatbots to answer customer questions. The second is to view the big data as observations about a complex system in our world and use machine learning and data mining techniques to discover hidden knowledge from the data and derive insights for optimization of decision making, thus enabling development assistive intelligent systems that can augment, rather than mimic, human intelligence.

The vision of the DataScope project is to develop a general big data analytics system (i.e., a "datascope") that can support a user in an interative way to analyze all kinds of data, including particularly natural language text data, which can be regarded as data reported by humans as subjective intelligent "sensors" of our world, as well as non-text data, which can be generated by (objective) physical sensors, for the purpose of knowledge discovery and decision optimization. Just as a microscope allows us to see things in the “micro world,” and a telescope allows us to see things far away, the envisioned DataScope would allow us to “see” useful hidden knowledge and patterns buried in all the data that we would otherwise not be able to see or see quickly/clearly, thus augmenting our intelligence in decision making.

The following pictures illustrate the vision of DataScope and how a DataScope would combine a wide range of techniques to provide interactive support for decision optimization, including, e.g., data understanding via natural language processing (NLP) and computer vision (CV), data integration to enable joint analysis of heterogeneous data from multiple sources, interactive retrieval and analysis of data to locate relevant data/information and discover latent patterns and knowledge, algorithms to support a user in hypothesis exploration and predictive modeling with aid of any domain knowledge, and user-friendly and personalized user interfaces to enable visualization, analytics workflow management, and intelligent adaptation to the needs of individual users to optimize the collaboration of DataScope with each individual user. DataScope augments human intelligence in decision making via prediction and quantification of the decision factors of uncertainties, which are often the main reason why a decision is difficult. The DataScope Project is meant to be a long-term "umbrella project" with many specific projects as special cases. More information about DataScope and specific projects in line of DataScope can be found from the following sources.

• A restricted initial version of the DataScope vision was presented in a SIGIR 2011 Keynote Talk by ChengXiang Zhai, Beyond Search: Statistical Topic Models for Text Analysis , where a vision of extending a search engine in a general way to create an analysis engine was discussed with statistical language modeling techniques, particularly probabilistic topic models, as building blocks. More detailed discussion of this vision can be found in the last chapter of the Text Data Management and Analysis book (Chapter 20: Toward a Unified System for Text Management and Analysis).
• With a focus on text data, the vision of TextScope, along with our work on joint analysis of text and non-text data, was later discussed in the following two talks: Keynote at IEEE BigData'17 , Distinguished Lecture at UNCC (2019), where DataScope was discussed as a more general vision of TextScope.
• Applications of DataScope: 1) Healthcare and Medicine: Medical DataScope; 2) IT Operation Optimization: DataScope for IT Operations (AIOps); 3) Education: DataScope for Learning Support (AI for Education); 4) Research Discovery: News Analytics System for Social Science Research, Gene Fitness Prediction for Biosynthesis, Leveraging knowledge networks for computational nutrition study.