A real web scraping app showed where AI automation gets hard
A local event app was built to gather live music, theater, sports, and similar listings from venue websites. GPT-5.5 was used to plan the approach, and VS Code was used to build a Python app with a . The workflow used Firecrawl to scan a site, GPT-4.x to choose pages that probably contained event details, Firecrawl again to read those chosen pages, and GPT-4.x to extract event information from the markdown output.
Real websites were harder than expected because their pages are inconsistent, and writing a reliable prompt for event extraction took real effort. Development happened with Codex inside VS Code on a Debian Linux using Gnome or Cinnamon. After about 10 hours, the app was good enough to run and stored events in a local database for later analysis, such as choosing what to do on the weekend.
Without past software development experience, this kind of project may be difficult to finish or not worth the time and cost.
Key points
- The app combined Firecrawl and GPT-4.x in several steps instead of sending every page straight to the model.
- GPT-4.x first helped narrow down which pages were likely to contain event information.
- The hardest part was dealing with messy websites and making a prompt that extracted events reliably.
- The finished version stored event data in a local database for later analysis.
- Past software development experience made the project more realistic and cost-effective.