On May 14th the nfl public data feed went dark. We do not know why they went down. We do not know if they will ever be back. However, a week later it’s apparant we should count on this being gone. This has caused nflgame and many other open source projects to break. The maintainers of these projects have reached out to me and we are all assessing our options for waht to do next.
What’s to be done
- I’ve reached out to several people at the NFL without any reply. There are several others with direct connections and we are waiting for advice from them.
- I have a release planned for Next Friday (5/30) that will make nflgame functional with historic data.
- Write a generic scraper to get all of the data that nfl consumes. Planned for mid June (unless you want to contribute).
Potential changes in data tracked
It appears the nfl went through with some major changes in how their data is presented. This will potentially be affecting how/what data is being stored. It’s entirely possible that we get some new data feed by the new year, OR some kind of officially sanctioned way to get at this data.
There are some scripts being worked on in the community that use selenium wire to intercept the api calls. This is tricky because it’s possibly against the terms. We are allowed to scrape data if we dont use it for commercial gain (12b1 in the terms). But… also (the second) 12a in those terms says “accessing data not intended for you or logging into a server or account that you are not authorized to access”.
So we will go ahead and scrape these api calls… asking for forgivness because we dont even know who to ask permission from. Are you connected w/ the NFL api team? NFL in any way? Reach out… lets talk about this.
Resuming Regular Posts
Between this and looking for work, it’s been unrealistic for me to continue writing. I will be resuming my regular scheduled usually-on-monday-but-sometimes-tuesday-or-at-the-latest-wednesday posts.
Currently I’m planning on taking a step back and figuring out why
npm audit was stating it was scanning 900k packages with create react app, this week its only 1500. I had planned on seriously GUTTING create react app, but it was actually pretty trivial to upgrade from 2.x to 3.x. Unsure what next weeks post will be about, but it will be amazing so you should stick around!