All work

Project 07 / 08Data Science

SAE – Twitter Data Processing (BUT Year 1)

Collection, processing and analysis of tweets using Python to identify trends by time periods.

Context
Coursework · Year 1
Field
Data & vision
Tools & technologies
  • Python
  • Pandas
  • Matplotlib
  • Data Analysis
  • CSV
  • JSON
  • Statistics
  • Visualization
Twitter data analysis

01

Project Objective

As part of this SAE, we were tasked with collecting, processing and analyzing Tweets using various digital tools. Our goal was to determine trends by periods based on the occurrences of keywords and display these trends visually with graphs.

02

Project Steps

  • Dataset retrieval: Collection of tweets in CSV, JSON and TXT formats
  • File parsing: Learning to read and write files in Python, handling different data formats
  • Data cleaning: Duplicate removal, spelling correction, filtering relevant content
  • Occurrence rate calculation: Counting specific keyword occurrences across different time periods
  • Statistical analysis: Identifying trends and correlations in the data
  • Visualization with Matplotlib: Creating graphs (curves, histograms, scatter plots) to display trends clearly and impactfully

03

Technologies & Tools Used

Python
Main language for processing and analysis
Pandas
Structured data manipulation and cleaning
Matplotlib
Creating graphical visualizations of trends
CSV/JSON/TXT Files
Parsing and processing multiple formats

04

Skills Acquired

Data Manipulation with Python
Proficiency with Pandas for cleaning and transforming large amounts of data
File I/O Operations
Parsing CSV, JSON and TXT files in Python
Statistical Analysis
Calculation of occurrence rates, frequencies and correlations
Data Visualization
Graphical storytelling with Matplotlib to present trends impactfully
Data Cleaning
Handling duplicates, error correction and data preparation