M4 Reflection

Considering what you now know about web scraping, what are some ways you can imagine using it? And what about FOIA? Have you ever submitted a FOIA request? If so, for what? If not, what’s something you like to FOIA and why?


  1. Considering what you now know about web scraping, what are some ways you can imagine using it? And what about FOIA? Have you ever submitted a FOIA request? If so, for what? If not, what’s something you like to FOIA and why?

    As, I’ve stated before, the technology aspect of my job is very new to me, but learning some of these tools has been both frustrating and exhilarating. In watching Paul Bradshaw’s scraping lecture I was amazed at how some of these techniques can really help tell the story. Seeing the corporatizing of Olympic torch-bearing was disheartening, but scraping that data helped make the case. 

    I work in sports and before Nate Silver was a rockstar in data-journalism, he was a quirky baseball guy leading the information revolution. I used to put him on when I was a producer and I would sit there-mouth agape as he would explain what I was looking at on the baseball diamond. Since those early days in 2003, I’ve been trying to marry the objective with the subjective in sports. It’s hard. There are plenty of people who are adverse to adding objective statistical analysis to sports.

    These fans have had their worlds flipped upside down, learning that the heritage stats that they’re used to: Batting Average, Pitcher Wins, RBI, etc…matter a little bit less. What I love about Silver is that he makes it a point to get to the context of the numbers. One doesn’t have to know the math, to understand it. I actually had one of his disciples, Chris Herring, on my show a couple of weeks ago and we had a big conversation about using stats in the proper context: (February 2nd, 19 minutes in http://chicago.cbslocal.com/audio/laurence-holmes/).

    I’ve never had to submit a FOIA request, but I came close a few years ago. http://www.washingtonpost.com/wp-dyn/content/article/2006/12/16/AR2006121600392_pf.html

    When I was a beat reporter for the Bears one of the players best friends was murdered at a club. I thought I might have to order a FOIA for the coroner, but I had a source at the coroner’s office that gave me good info. Learning how to do FOIA request should be required for everyone who enters the newsroom. It’s a skill that has incredible benefits. 

    1. Thanks for sharing the link to the show, Laurence. Chris makes a great point about how data can muddy a story instead of help clarify it. Like with maps, data (and data viz) must be deployed well. So cool that you booked a young Nate Silver! My time at 538 is clustered around national elections; I need to start reading more of their cultural and sports coverage.

  2. I had never heard of data scraping. It seems very cool and useful, especially combining it with Google Fusion Tables. It’s so easy to turn the data into charts, and I’m assuming you can use scraped data to create maps too if there is address information.

    There are obviously many uses for scraping, but I can see it being used where there is a lot of statistical data, like the Kickstarter page we used for our assignment. You could scrape census data, polling data, etc. and manipulate and visualize them in a lot of interesting ways, not necessarily just using the one-click chart tools.

    I’ve never submitted a FOIA request. I haven’t been in a position where I needed to, so I don’t really know of anything that I would like to FOIA. When I was covering the Northport City Council for a class, I might have requested documents for something like the departure of the city administrator, but that wouldn’t fall under FOIA since it’s not a federal agency.

    1. All states have so-called “sunshine laws” that require agencies to hold open meetings and make available certain documents. Alabama’s law reads: “Every citizen has a right to inspect and take a copy of any public writing of this state, except as otherwise expressly provided by statute.” That last part — “except as otherwise expressly provided by statute” — is the complicating factor. The Reporters Committee for Freedom of the Press has a helpful guide to each state’s open-meeting and open-records law (https://www.rcfp.org/open-government-guide).

  3. I actually heard of data scraping at the beginning of this semester in a graduate theatre class where we are designing a marketing plan for Opera Birmingham. To design a marketing plan, you have to be able to research and pull data off of many different websites, and our instructor was a huge fan of data mining. Although I had not actually had to perform scraping until this class, it was a nice little coincidence that we went over that in class. In the past two days, I have actually used this process to pull financials and patron statistics off of websites and onto spreadsheets so that I could more easily work with those documents and sort the information. It has been a very valuable asset to the project.

    In terms of using this when I finally get a job in education, I think I could see myself using it for instruction purposes, like if I needed to show students some tricks to analyzing data for research papers and what not. I think it would be harder to adopt something like this into an English classroom, but there are probably some ways that I have not considered yet. I do know that if I end up having to coach a sport as a teacher, this would probably be a pretty good tool to have just in case I needed to go online and pull rosters or stats. Then it could really be useful to me.

    As far as submitting a FOIA, I have never had to do this. While I have no specifics in mind that I would like to FOIA, I think it would still be good to have for my future as an educator. As schools continue to become more and more politicized, it would definitely benefit me to use this skill at some point in my career. Right now, for instance, with the teacher tax cut, there is a lot of talk that insurance will rise and offset those savings. It is currently a big deal in my field, and maybe a FOIA could help shed some more light on the situation.

    1. Glad to hear about the scraping alignment. I just helped organize a read-a-thon, for which we read short stories out loud and nonstop for 24 hours. In past years, we’ve read novels, and it’s easy enough to schedule those because of online lists that compile word counts for famous novels (e.g., http://commonplacebook.com/art/books/word-count-for-famous-novels/). But short stories were a lot harder to schedule, and I found myself wishing for a giant database of short stories and word counts. Some day…

Leave a Reply