DSCI 550: Data Science at Scale

Class Info

Spring Semester, 2021
: OHE 100 and online
: Th 3:30-6:50pm
: 32413D
: 32426D (DEN)

Instructor



http://irds.usc.edu/faculty/mattmann/
 chris.a.mattmann@jpl.nasa.gov
 2:30pm-3:30pm   (right before class)

Teaching Assistant

Keerti Bhogaraju

E-Mail: bhogaraj@usc.edu
 Friday 12pm to 2pm  

DSCI 550 Overview

This course is designed as an overview course to give students a broad understanding of Informatics topics for Big Data and to get practical experience with key Big Data informatics techniques. Topics include roadmap of informatics, the data lifecycle, the role of the data scientist, and analyzing and exploring Big Data with real world use cases in data analytics, and big data. Understanding Big Data involves understanding of digital file formats, their detection and data extraction from them. Emphasis areas include Document Type Detection; Parsing and extraction; Metadata understanding and analysis; Language Identification and detection from files and finally file formats and representation. The class also has a specific focus on Content Detection and Analysis from large data sets. Datasets used in the course are publicly collected by the instructor or his collaborators involved in national Big Data initiatives including DARPA, NASA and other projects. The course is designed to be accessible to students with experience programming in Python and Java at an intermediate level. The course will introduce the students to topical software frameworks that deal with Big Data including Tika, Solr, ElasticSearch™, TensorFlow, Nutch and Apache Hadoop™. The course will be a combination of lecture, in-class discussion, readings, group-based assignments and a final exam.

The objective of this course is to train students to be able to understand Big Data and Large Data Environments, e.g., file formats, their representation, and how to automatically extract information from large datasets of files. Specifically, students successfully completing this course will achieve three main objectives:

  1. Develop sufficient proficiency in Big Data frameworks to write software capable of automatically extracting information from data including its text and metadata and language.
  2. Develop sufficient proficiency in techniques with Large Data sets collected from the Web and other places (Intranet, Science Data Sets, Public Data Sets).
  3. Develop sufficient proficiency in Python and Java to write and execute software that is “File Aware” and that automatically extracts text and metadata from large data sets.

The primary teaching methods will be discussion, case studies, and lectures. Students are expected to perform directed self learning outside of class which encompasses, among other things, a considerable amount of literature review. In addition, the class will directly leverage open source software and partnerships from the Instructor is a former member of the Board of Directors at the Apache Software Foundation. Projects associated with the course make direct contributions to Apache Licensed (“ALv2”) open source software projects at the student’s discretion. Leadership training in open source is provided and encouraged, and students leave with an experience in open source that makes them more marketable to companies and institutions looking to hire in Big Data, and Data Science.

In addition to foundations, and practical experience with Big Data and Data Science, the class will also introduce the student to the state-of-the-art in content detection research, future trends and state-of-the-practice. Students are expected to attend class regularly, and participate (as directed) in all class discussions, and most importantly, have fun!

USC ACADEMIC INTEGRITY

Statement on Academic Conduct and Support Systems

Academic Conduct Plagiarism - presenting someone else.s ideas as your own, either verbatim or recast in your own words - is a serious academic offense with serious consequences. Please familiarize yourself with the discussion of plagiarism in SCampus in Section 11, Behavior Violating University Standards. Other forms of academic dishonesty are equally unacceptable. See additional information in SCampus and university policies on scientific misconduct. Discrimination, sexual assault, and harassment are not tolerated by the university. You are encouraged to report any incidents to the Office of Equity and Diversity or to the Department of Public Safety. This is important for the safety whole USC community. Another member of the university community - such as a friend, classmate, advisor, or faculty member - can help initiate the report, or can initiate the report on behalf of another person. The Center for Women and Men provides 24/7 confidential support, and the sexual assault resource center webpage sarc@usc.edu describes reporting options and other resources.

Support Systems

A number of USC's schools provide support for students who need help with scholarly writing. Check with your advisor or program staff to find out more. Students whose primary language is not English should check with the American Language Institute which sponsors courses and workshops specifically for international graduate students. The Office of Disability Services and Programs provides certification for students with disabilities and helps arrange the relevant accommodations. If an officially declared emergency makes travel to campus infeasible, USC Emergency Information will provide safety and other updates, including ways in which instruction will be continued by means of blackboard, teleconferencing, and other technology.

Statement on Diversity

The diversity of the participants in this course is a valuable source of ideas, problem solving strategies, and engineering creativity. We encourage and support the efforts of all of our students to contribute freely and enthusiastically. We are members of an academic community where it is our shared responsibility to cultivate a climate where all students and individuals are valued and where both they and their ideas are treated with respect, regardless of their differences, visible or invisible.

TEXTBOOK

Chris A. Mattmann, and Jukka Zitting. Tika in Action, 256 pages. New York: Manning Publications, November 2011. ISBN: 9781935182856.

 

C. Mattmann. Machine Learning with TensorFlow: 2nd Edition. 456 pages. New York: Manning Publications, December 2020. ISBN 9781617297717 

ASSIGNMENTS and EXAMINATIONS

Name Description Weight
Exam An exam testing your understanding of the lecture materials 25%
Assignments Assignments where you will build on the Big Data and Data Science topics in course and make a contribution to one of the existing open source frameworks (Tika, Solr, Nutch, TensorFlow, OODT, etc.). 45%
Individual Presentation An individual presentation demonstrating the student's understanding of one of the required paper readings in the course. 25%
Participation Participation in lectures, by asking questions and contributing to the conversation. Attending lectures (physically and remotely) and positively contributing to the class experience. 5%

Project Submission Guidelines

Submission guidelines will be specified in each assignment.

Schedule (subject to change; check regularly)

Week Lecture Topic Assigned Readings Assignments & Deadlines
1

(Jan 21st, 2021)
  • Course Introduction
  • Introduction to Big Data
  • DARPA XDATA Program - Overview Slides
  • Breakout Groups on Big Data
  • Tika in Action, Chapter 1
  • Mattmann, Chris. A vision for data science. Nature, Vol. 493, No. 7433, pp. 473-475, January 24, 2013.
  • Lynch, Clifford. "Big data: How do your data grow?." Nature 455.7209 (2008): 28-29. (Presentation by: Reid Pattis)
  • Howe, Doug, et al. "Big data: The future of biocuration." Nature 455.7209 (2008): 47-50.(Presentation by: Xiaoyu Wang)
  • Wigan, Marcus R., and Roger Clarke. "Big data's big unintended consequences." Computer 46.6 (2013): 46-53.
  • Schwartz, J. A. N. A., et al. "Measuring the value of Big Data exploitation systems: Quantitative, non-subjective metrics with the user as a key component." Parsons Journal for Information Mapping 6 (2014): 1-12.
  • Sotera Defense Solutions. A Survey of Big Data Methods, Assessments, and Approaches. November 2012 (Presentation by: Peizhen Li)
  • De Mauro, Andrea, Marco Greco, and Michele Grimaldi. "What is big data? A consensual definition and a review of key research topics." AIP conference proceedings. Vol. 1644. No. 1. AIP, 2015. (Presentation by: Mike Hanono)
 Resources:

  • DARPA I2O (DARPA Dan) Video
  • Zero Dark Thirty Workbench Video
2

(Jan 28th, 2021)
  • Report out from Big Data Breakouts
  • A Taxonomy of File Formats
  • Content Detection Libraries
  • Language Bindings for Apache Tika
  • Individual Student Presentations - Week 1 Papers
  • Tika in Action, Chapter 2
  • Crocker, David. RFC 822 "Standard for the format of ARPA Internet text messages." (1982).(Presentation by: Mengxun Li)
  • Freed, Ned and Nathaniel Borenstein. RFC 1341. MIME (Multipurpose Internet Mail Extensions). Mechanisms for Specifying and Describing the Format of Internet Message Bodies. June 1992. (Presented by: Gabby Jia)
  • Freed, Ned, and Nathaniel Borenstein. RFC 2045. Multipurpose internet mail extensions (MIME) part one: Format of internet message bodies. 1996.(Presentation by: Tricia John)
  • Freed, Ned, and Nathaniel Borenstein. RFC 2046 Multipurpose internet mail extensions (MIME) part two: Media types, November, 1996.(Presented by: Dongzhe Jiang)
  • Freed, Ned. RFC 2048 "Multipurpose internet mail extensions (MIME) part four: Registration procedures." ISI (1996). (Presented by: )
  • Hicks, Ben J., et al. "Organizing and managing personal electronic files: A mechanical engineer's perspective." ACM Transactions on Information Systems (TOIS) 26.4 (2008): 23.
  • Shim, Jungwon Roy. "Arium: Beyond the Desktop Metaphor: A new way of navigating, searching, and organizing personal digital data." Masters Thesus, Carnegie Mellon University (2012).(Presented by: Zihao Niu)
  • Crowder, Jerome, Jonathan Marion, and Michele Reilly. "File Naming in Digital Media Research: Examples from the Humanities and Social Sciences." Journal of Librarianship and Scholarly Communication 3.3 (2015). (Presented by: Shiyun Zheng)
  • Jackson, Andrew N. "Formats over time: Exploring UK web history." arXiv preprint arXiv:1210.1714 (2012). (Presentation by: Yue Meng)
  
3

(Feb 4th, 2021)
  • Report outs from the in class discussion around classifying files and the MIME taxonomy
  • Document Similarity and Deduplication
  • Individual Presentations - Week 2 Papers
  • Tika in Action, Chapter 3
  • Bik, Elisabeth M., Casadevall, Arturo, Fang, Ferrie C. The Prevalence of Inappropriate Image Duplication in Biomedical Research Publications.(Presentation by: Saumya Shah)
  • Manku, Gurmeet Singh, Arvind Jain, and Anish Das Sarma. "Detecting near-duplicates for web crawling." Proceedings of the 16th international conference on World Wide Web. ACM, 2007.(Presented by: Zhou Daisy)
  • Henzinger, Monika. "Finding near-duplicate web pages: a large-scale evaluation of algorithms." Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 2006.(Presented by: Melina Jiang )
  • Cooper, Matthew, Jonathan Foote, and Andreas Girgensohn. "Automatically organizing digital photographs using time and content." Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on. Vol. 3. IEEE, 2003.
  • Manber, Udi. "Finding similar files in a large file system." Usenix Winter. Vol. 94. 1994. (Presented by: Tianfeng Huang)
  • Chim, Hung, and Xiaotie Deng. "Efficient phrase-based document similarity for clustering." IEEE Transactions on Knowledge and Data Engineering 20.9 (2008): 1217-1229.
Resources: 
4

(Feb 11th, 2021)
  • Remaining presentations from last week (if any)
  • Document Type Detection
  • Individual Presentations - Week 3 papers
  • Advanced File System Statistics and Understanding
  • Tika in Action, Chapter 4
  • Amirani, Mehdi Chehel, Mohsen Toorani, and A. Beheshti. A new approach to content-based file type detection. Computers and Communications, 2008. ISCC 2008. IEEE Symposium on. IEEE, 2008 (Presented by: Qing Wei)
  • McDaniel, Mason, and M. Hossain Heydari. Content based file type detection algorithms. System Sciences, 2003. Proceedings of the 36th Annual Hawaii International Conference on. IEEE, 2003.(Presented by: Yichun Yuan)
  • Alamri, Nasser S., and William H. Allen. "A comparative study of file-type identification techniques." SoutheastCon 2015. IEEE, 2015.(Presentation by: Yueqi Gu)
  • Li, Wei-Jen, et al. "Fileprints: Identifying file types by n-gram analysis." Information Assurance Workshop, 2005. IAW'05. Proceedings from the Sixth Annual IEEE SMC. IEEE, 2005.
  • Shahi, Ashim. "Classifying the classifiers for file fragment classification." Masters Thesis, Universiteit van Amsterdam (2012). (Presentation by: Mohan Xing)
  • Ahmed, Irfan, et al. "Fast file-type identification." Proceedings of the 2010 ACM Symposium on Applied Computing. ACM, 2010. (Presentation by: Peiran Du)
  • Pierris, Georgios, and Stilianos Vidalis. "Forensically classifying files using HSOM algorithms." Emerging Intelligent Data and Web Technologies (EIDWT), 2012 Third International Conference on. IEEE, 2012.(Presentation by:  Supraja Krovvidi)
  • Harris, Ryan M. "Using artificial neural networks for forensic file type identification." Master's Thesis, Purdue University (2007). (Presentation by: Sarah Pursley)
  • Douceur, John R., and William J. Bolosky. A large-scale study of file-system contents. ACM SIGMETRICS Performance Evaluation Review 27.1 (1999): 59-70. (Presentation by: Katie Chak)
 Resources: 
5

(Feb 18th, 2021)
  • Introduction to Assignment 1
  • Content Extraction
  • Individual Presentations - Week 4 papers
  • Tika in Action, Chapter 5
  • Kilicoglu, Halil, et al. "Semantic MEDLINE: a web application for managing the results of PubMed Searches." Proceedings of the third international symposium for semantic mining in biomedicine. Vol. 2008. 2008.(Presentation by: )
  • Kobayashi, Mei, and Koichi Takeda. "Information retrieval on the web." ACM Computing Surveys (CSUR) 32.2 (2000): 144-173. (Presentation by: Yue Liu)
  • Voorhees, Ellen M., and Donna Harman. "Overview of the sixth text retrieval conference (TREC-6)." Information Processing & Management 36.1 (2000): 3-35. 
  • Arasu, Arvind, and Hector Garcia-Molina. Extracting structured data from web pages. Proceedings of the 2003 ACM SIGMOD international conference on Management of data. ACM, 2003.(Presented by: Kehan Bi)
  • Lewandowski, Dirk. "Web searching, search engines and Information Retrieval." Information Services & Use 25.3, 4 (2005): 137-147. (Presented by: Shuying Ye)
  • Weninger, Tim, William H. Hsu, and Jiawei Han. "CETR: content extraction via tag ratios." Proceedings of the 19th international conference on World wide web. ACM, 2010. (Presentation by: Mengjie Deng)
  • Karpathy, Andrej, and Li Fei-Fei. "Deep visual-semantic alignments for generating image descriptions." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.(Presentation by: Xiuwen Zhang)
 Resources: 
6

(Feb 25th, 2021)
  • Assignment 1 questions 
  • Last Week Tonight:  Robocalls
  • RoboKiller Video
  • Understanding Metadata
  • Individual Presentations - week 5 papers
  • Tika in Action, Chapter 6
  • Gowda, Thamme, and Chris A. Mattmann. "Clustering Web Pages Based on Structure and Style Similarity (Application Paper)." Information Reuse and Integration (IRI), 2016 IEEE 17th International Conference on. IEEE, 2016.(Presentation by: Nat Zheng)
  • Anquetil, Nicolas, and Timothy Lethbridge. File clustering using naming conventions for legacy systems. Proceedings of the 1997 conference of the Centre for Advanced Studies on Collaborative research. IBM Press, 1997.(Presentation by: Peng Rui)
  • Swierk, Edward, et al. "The Roma personal metadata service." Mobile Networks and Applications 7.5 (2002): 407-418. (Presentation by: Claudia Winarko)
  • Karypis, Michael Steinbach George, Vipin Kumar, and Michael Steinbach. "A comparison of document clustering techniques." KDD workshop on Text Mining. 2000.(Presented by: Xuemin Zeng)
  • Marchionini, Gary. "Exploratory search: from finding to understanding." Communications of the ACM 49.4 (2006): 41-46. (Presentation by: Jacob Bickman)
 Resources:
7

(March 4th, 2021)
  • Video - Understanding Metadata TedX talk
  • Information Clustering
  • Example - Machine Learning with TensorFlow - Activity Clustering (Android)
  • Week 6 Individual Presentations
  • Tika in Action, Chapter 7
  • Koehn, Philipp, et al. "Moses: Open source toolkit for statistical machine translation." Proceedings of the 45th annual meeting of the ACL on interactive poster and demonstration sessions. Association for Computational Linguistics, 2007.(Presentation by: Jason Davenport )
  • Post, Matt, et al. "Joshua 5.0: Sparser, better, faster, server." Proceedings of the Eighth Workshop on Statistical Machine Translation. 2013.(Presentation by: )
  • Lins, Rafael Dueire, and Paulo Gonçalves. Automatic language identification of written texts. Proceedings of the 2004 ACM symposium on Applied computing. ACM, 2004.(Presentation by: Amber Yu)
  • Papineni, Kishore, et al. "BLEU: a method for automatic evaluation of machine translation." Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics, 2002. (Presentation by: Sungho Lee )
  • Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. "Neural machine translation by jointly learning to align and translate." arXiv preprint arXiv:1409.0473 (2014). (Presentation by: Ruochen Cao)
  • Tromp, Erik, and Mykola Pechenizkiy. "Graph-based n-gram language identification on short texts." Proc. 20th Machine Learning conference of Belgium and The Netherlands. 2011. (Presentation by: Xinran Liang )
  • Lopez-Moreno, Ignacio, et al. "Automatic language identification using deep neural networks." Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on. IEEE, 2014.(Presentation by: Jiarui Ou )
  • Bertoldi, Nicola, et al. "MMT: New open source MT for the translation industry." Proceedings of The 20th Annual Conference of the European Association for Machine Translation (EAMT). 2017.
Resources: 
8

(March 11th, 2021)
  • Exam Review
  • Assignment 2
  • Video - Linguistic Forensics
  • Language Identification
  • Week 7 Individual Presentations
  • Tika in Action, Chapter 8
  • Tjong Kim Sang, Erik F., and Fien De Meulder. "Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition." Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4. Association for Computational Linguistics, 2003.(Presented by: Yadi Zhao)
  • Nadeau, David, and Satoshi Sekine. "A survey of named entity recognition and classification." Lingvisticae Investigationes 30.1 (2007): 3-26.(Presentation by: Madeleine Thompson )
  • Ritter, Alan, Sam Clark, and Oren Etzioni. "Named entity recognition in tweets: an experimental study." Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2011.
  • Mattmann, Chris A., and Madhav Sharan. "An automatic approach for discovering and geocoding locations in domain-specific web data." Proceedings of the 2016 IEEE 17th International Conference on Information Reuse and Integration (IRI’16). 2016. (Presentation by: Yifeng Shi )
  • Khodak, Mikhail, Nikunj Saunshi, and Kiran Vodrahalli. "A Large Self-Annotated Corpus for Sarcasm." arXiv preprint arXiv:1704.05579 (2017).(Presentation by: Ben Gumser )
  • Hutto, Clayton J., and Eric Gilbert. "Vader: A parsimonious rule-based model for sentiment analysis of social media text." Eighth international AAAI conference on weblogs and social media. 2014.(Presented by: Hanyu He)
  • Geyer, Kelly, et al. "Named Entity Recognition in 140 Characters or Less." # Microposts. 2016. (Presentation by: Aparna Nair )
Resources:
9

(March 18th, 2021)
  • Exam 
  • Spillover lecture - Language Identification 
  • Individual Presentations
 Location:  Online Resources:
10

(March 25th, 2021)
  • Individual Presentations
  • Discussion on Named Entity Recognition
  • Hadoop Spark and Tika: Large Scale Content Detection and Analysis
  • Tika in Action, Chapter 9
  • Dean, Jeffrey, and Sanjay Ghemawat. MapReduce: simplified data processing on large clusters. Communications of the ACM 51.1 (2008): 107-113.(Presentation by: Shruti Krishna Kumar)
  • Zaharia, Matei, et al. Spark: cluster computing with working sets.Proceedings of the 2nd USENIX conference on Hot topics in cloud computing. Vol. 10. 2010.(Presented by: Qingyu Chen)
  • Elsayed, Tamer, Jimmy Lin, and Douglas W. Oard. "Pairwise document similarity in large collections with MapReduce." Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers. Association for Computational Linguistics, 2008. (Presentation by: Shoujie Chai)
  • M. Bernaschi, M. Cianfriglia, A. Di Marco, A. Sabellico, G. Me, G. Carbone, G. Totaro. Forensic Disk Image Indexing and Search in an HPC environment. IEEE International Conference on High Performance Computing & Simulation (HPCS), 2014.(Presented by: Meredith Brindley)
  • Meusel, Robert, Peter Mika, and Roi Blanco. "Focused crawling for structured data." Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. ACM, 2014.(Presented by: Fang Gwen )
  • Niu, Feng, et al. "DeepDive: Web-scale Knowledge-base Construction using Statistical Learning and Inference." VLDS 12 (2012): 25-28.(Presentation by: Xuxin Peng)
  • Mattmann, C. A., Oh, J. H., Palsulich, T., McGibbney, L. J., Gil, Y., & Ratnakar, V. (2015, November). DRAT: An Unobtrusive, Scalable Approach to Large Scale Software License Analysis. In Automated Software Engineering Workshop (ASEW), 2015 30th IEEE/ACM International Conference on (pp. 97-101). IEEE. (Presentation by: MacKenzie McClung)
 Resources:
11

(April 1st, 2021)
  • Assignment 3 - Introduction
  • Readout - Named Entity Recognition Group Presentations
  • Open Source Content Detection Technologies
  • Individual Presentations
  • Tika in Action, Chapter 10
  • Białecki, Andrzej, et al. "Apache lucene 4." SIGIR 2012 workshop on open source information retrieval. 2012.(Presented by: Anthony Zhang)
  • Turtle, Howard, Yatish Hegde, and S. Rowe. "Yet another comparison of lucene and indri performance." SIGIR 2012 Workshop on Open Source Information Retrieval. 2012.(Presented by: Olivia Fryt)
  • Bontcheva, Kalina, et al. "TwitIE: An Open-Source Information Extraction Pipeline for Microblog Text." RANLP. 2013.(Presentation by: Jiahao Sun )
  • Cunningham, Hamish. "GATE, a general architecture for text engineering." Computers and the Humanities 36.2 (2002): 223-254.(Presentation by: Dayuan Jian)
  • Atserias, Jordi, et al. "FreeLing 1.3: Syntactic and semantic services in an open-source NLP library." Proceedings of LREC. Vol. 6. 2006. (Presentation by: )
  • Manning, Christopher D., et al. "The stanford corenlp natural language processing toolkit." ACL (System Demonstrations). 2014.(Presentation by:  )
  • Savova, Guergana K., et al. "Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications." Journal of the American Medical Informatics Association 17.5 (2010): 507-513. (Presented by: Chuqi Liu)
 Resources:
12

(April 8th, 2021)
  • Ted Talk (click link in resources)
  • Evaluating Content Detection
  • Walkthrough of Polar Deep Insights
  • Individual Presentations
  • Tika in Action, Chapter 11
  • Nowell, Lucy Terry, et al. "Visualizing search results: some alternatives to query-document similarity." Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 1996. (Presentation by: Ming Yan )
  • Shneiderman, Ben. "The eyes have it: A task by data type taxonomy for information visualizations." Visual Languages, 1996. Proceedings., IEEE Symposium on. IEEE, 1996.(Presentation by: Leon Luc)
  • Gottron, Thomas. "Evaluating content extraction on HTML documents." Proceedings of the 2nd International Conference on Internet Technologies and Applications (ITA’07). 2007.
  • Leuski, Anton. "Evaluating document clustering for interactive information retrieval." Proceedings of the tenth international conference on Information and knowledge management. ACM, 2001.
  • Bailey, Peter, et al. "Evaluating search systems using result page context." Proceedings of the third symposium on Information interaction in context. ACM, 2010.(Presentation by: Benjamin Chan )
 Resources:
13

(April 15th, 2021)
  • Group Readouts on Evaluating Content Detection and Analysis
  • Lecture on NoSQL
  • Lecture on SciSpark
  • Individual Presentations
  • Palamuttam, Rahul, et al. "SciSpark: Applying in-memory distributed computing to weather event detection and tracking." Big Data (Big Data), 2015 IEEE International Conference on. IEEE, 2015.(Presentation by:)
  • Leavitt, Neal. "Will NoSQL databases live up to their promise?." Computer 43.2 (2010).(Presented by: Yixuan Wang)
  • Stonebraker, Michael. "SQL databases v. NoSQL databases." Communications of the ACM 53.4 (2010): 10-11.(Presentation by: Meihui Chen)
  • Stonebraker, Michael. "Stonebraker on NoSQL and enterprises." Communications of the ACM 54.8 (2011): 10-11.
  • Rafique, Ansar, et al. "On the performance impact of data access middleware for nosql data stores." IEEE Transactions on Cloud Computing (2015).(Presentation by: Shikun Gou)
  • Moniruzzaman, A. B. M., and Syed Akhter Hossain. "Nosql database: New era of databases for big data analytics-classification, characteristics and comparison." arXiv preprint arXiv:1307.0191 (2013). (Presentation by: Lidan Cui)
Resources:
14

(April 22nd, 2021)

 No Class This Week No Class This Week - Student Wellness Day (April 22nd, 2021)  
15

(April 29th, 2021)
  • Video - Scientific Data: Water and Snow in the Western US
  • Searching Scientific Datasets
  • Scientific Data Processing (Airborne Snow Observatory)
  • Individual Presentations
  • Big Data with an Eye Towards the Future: Discussion 
  • Tika in Action, Chapter 12 - 14
  • C. Mattmann, D. Freeborn, D. Crichton, B. Foster, A. Hart, D. Woollard, S. Hardman, P. Ramirez, S. Kelly, A. Y. Chang, C. E. Miller. A Reusable Process Control System Framework for the Orbiting Carbon Observatory and NPP Sounder PEATE missions. In Proceedings of the 3rd IEEE Intl Conference on Space Mission Challenges for Information Technology (SMC-IT 2009), pp. 165-172, July 19 - 23, 2009. (Presentation by: Wenhao Li )
  • Wilkinson, Mark D., et al. "The FAIR Guiding Principles for scientific data management and stewardship." Scientific data 3 (2016): 160018.(Presentation by:Zixi Jiang)
  • Buneman, Peter, et al. "Archiving scientific data." ACM Transactions on Database Systems (TODS) 29.1 (2004): 2-42.
  • Fox, Peter, and James Hendler. "Changing the equation on scientific data visualization." Science 331.6018 (2011): 705-708.(Presentation by: Xiang Li )
  • Plale, Beth, et al. "Active management of scientific data." IEEE Internet Computing 9.1 (2005): 27-34.(Presented by: Wenting Shang)
  • Gray, Jim, et al. "Scientific data management in the coming decade." ACM SIGMOD Record 34.4 (2005): 34-41. (Presented by: Xinyue Zhang)
  • Ailamaki, Anastasia, Verena Kantere, and Debabrata Dash. "Managing scientific data." Communications of the ACM 53.6 (2010): 68-78. (Presentation by: Yue Qi)
 Resources: