"Slapshot" and AI

We’ve just finished a milestone integration project using shiny AI for … web scraping. And it felt as honest and riveting as when that was the only practical approach available circa .com bubble.

Back in early 2000s we had a French team in charge of scraping banking websites. When I went to meet them in Palo Alto, their team lead introduced himself as “Claude - the lead SCRAPPER”. Those of you who remember the vintage "Slapshot" movie would appreciate how much this statement endeared Claude on the Canadian hockey crazed part of the team. The rules of engagement were simple - do whatever it takes to extract the data out of a website. Not much has changed, to be honest other than the scraping tools and the fact that nowadays one talks to another Claude to analyze a website structure and figure out a way to get the right info.

Well, the tech progress did bring a nice little improvement. When the integration went live, we’ve received an email from that platform we scraped and scrapped with. The vendor inquired about a highly unusual interaction pattern, got the honest “we had to get it done” answer and responded with “that’s what the APIs are for”.

The real rules are timeless - scrap first, then shake hands out of respect. Sometimes it even starts a long lasting friendship.

Übersetzung anzeigen

‍