Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Show HN: BigQuery dataset: 22M FCC comments on net neutrality (cloud.google.com)
119 points by sajacy on Sept 6, 2017 | hide | past | favorite | 18 comments


Looks like there are about 255,640-460,552 truly unique (i.e. hand-written) comments roughly (1-2%).

SELECT count(1) as cnt, substr(comment, 0,30) as abbrev_comment FROM [jyang:fcc_ecfs.17108_comments] GROUP BY abbrev_comment order by cnt DESC LIMIT 1000000


Often, when members of the public are encouraged to petition for something, those doing the encouraging also provide form letters to which the petitioner just has to add their signature.

That the contents are identical doesn't mean that real different people didn't send them.


Extremely valid observation, I would just like to add that the grandparent post referenced hand written specifically, which is just as important when the outcome isn't binary.


I think they meant "hand written" as "unique", rather than "written by hand on paper and posted"? The query they show is just counting the unique first 30 chars (and would count form letters less than 30 chars where the name on the end makes them unique too etc).


Yes this is true but it is not a productive practice nor helpful. These responses are supposed to be comments; well formed opinions and policy suggestions which policy makers can read to gain perspective. Its not a system for voting.

It is OK to think that US citizens should have a more direct, democratic say in the power structures that control the internet. But FCC comments can never be a good vehicle for this. To those currently upset at the FCC over the rollback of an Obama era rule, I would invite you to critically consider what parts of the communications acts are actually constitutional and whether photons superimposed across state lines should fall under the scope of the interstate commerce clause or the bill of rights.


1,525,564 records (6.8%) seemingly generated from https://www.battleforthenet.com/

> SELECT COUNT(*) FROM [jyang:fcc_ecfs.17108_comments] WHERE comment LIKE "The FCC's Open Internet Rules (net neutrality rules) are extremely important to me.%"


Most popular comment openings:

> SELECT COUNT(*) c, SUBSTR(comment, 0, 100) pre FROM [jyang:fcc_ecfs.17108_comments] GROUP BY pre ORDER BY c DESC LIMIT 10;


1 1,523,300 The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to pr

2 1,359,677 In 2015, Chairman Tom Wheeler’s Federal Communications Commission (FCC) imposed restrictive Title II

3 1,283,663 Before leaving office, the Obama Administration rammed through a massive scheme that gave the federa

4 958,487 The current FCC regulatory scheme known as "Title II" represents an unprecedented increase in govern

5 818,815 The unprecedented regulatory power the Obama Administration imposed on the internet is smothering in

6 657,153 As a concerned taxpayer and consumer, I am writing to urge the FCC to set the internet free and remo

7 531,981 The Obama-era FCC regulations known as "Title II" enable the federal government to exert an extraord

8 452,764 I am in favor of strong net neutrality under Title II of the Telecommunications Act. Sincerely, Ma

9 450,914 The Open Internet rules (net neutrality) are extremely important to me. I don't want the ISP to have

10 380,872 I am in favor of strong net neutrality under Title II of the Telecommunications Act. Sincerely, Jo


Huh, I wonder why the top negative copy+paste responses are not tied to any organization or grass-roots effort.


Presumably they must have come from somewhere, anyone know where?



From the looks of it, 8 and 10 are likely the same. Shrinking the preamble check by a few characters will likely make them show up better and get a better idea of the last few entries in the top 10.


Indeed, that small change brings it up to the top of the ranking!

1 | [7568642] | am in favor of strong net neutrality under Title II of the Telecommunications Act. Sincerely,

2 | [1523457] | The FCC's Open Internet Rules (net neutrality rules) are extremely important to me. I urge you to

3 | [1359677] | In 2015, Chairman Tom Wheeler's Federal Communications Commission (FCC) imposed restrictive Title

4 | [1283663] | Before leaving office, the Obama Administration rammed through a massive scheme that gave the fede

5 | [958487] | The current FCC regulatory scheme known as "Title II" represents an unprecedented increase in gove

6 | [818815] | The unprecedented regulatory power the Obama Administration imposed on the internet is smothering

7 | [657153] | As a concerned taxpayer and consumer, I am writing to urge the FCC to set the internet free and re

8 | [531981] | The Obama-era FCC regulations known as "Title II" enable the federal government to exert an extrao

9 | [450914] | The Open Internet rules (net neutrality) are extremely important to me. I don't want the ISP to ha

10 | [379644] | Obama's Federal Communications Commission (FCC) forced regulations on the internet that put the go


This could become An interesting dataset for distinguishing bots from people. Evidently wr need some kind of Voigt-Kampf test for social media.


Take the substr down and I think 8 and 10 combine and may include a very large number of additional matches.


I dont have a google account, can anyone re-post any of this?


Here's the preview offered of the table "17108_comments", in JSON format: http://sprunge.us/CeWh

There's also a table of throwaway email addresses and a view called "frequency_and_unique_comment_by_domain".

Consider getting a throwaway Google account, BigQuery's basically unusable without one.


All I get when opening this link is a "Welcome to BigQuery!" panel and a "Create a Project" button.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: