SAE – Twitter Data Processing (BUT Year 1)
Collection, processing and analysis of tweets using Python to identify trends by time periods.
- Context
- Coursework · Year 1
- Field
- Data & vision
- Tools & technologies

01
Project Objective
As part of this SAE, we were tasked with collecting, processing and analyzing Tweets using various digital tools. Our goal was to determine trends by periods based on the occurrences of keywords and display these trends visually with graphs.
02
Project Steps
- Dataset retrieval: Collection of tweets in CSV, JSON and TXT formats
- File parsing: Learning to read and write files in Python, handling different data formats
- Data cleaning: Duplicate removal, spelling correction, filtering relevant content
- Occurrence rate calculation: Counting specific keyword occurrences across different time periods
- Statistical analysis: Identifying trends and correlations in the data
- Visualization with Matplotlib: Creating graphs (curves, histograms, scatter plots) to display trends clearly and impactfully
03
Technologies & Tools Used
- Python
- Main language for processing and analysis
- Pandas
- Structured data manipulation and cleaning
- Matplotlib
- Creating graphical visualizations of trends
- CSV/JSON/TXT Files
- Parsing and processing multiple formats
04
Skills Acquired
- Data Manipulation with Python
- Proficiency with Pandas for cleaning and transforming large amounts of data
- File I/O Operations
- Parsing CSV, JSON and TXT files in Python
- Statistical Analysis
- Calculation of occurrence rates, frequencies and correlations
- Data Visualization
- Graphical storytelling with Matplotlib to present trends impactfully
- Data Cleaning
- Handling duplicates, error correction and data preparation