.Net developer, Student, and former Tenor. Interested in F# and Mono development.
3 stories

Here to Help

6 Comments and 23 Shares
"We TOLD you it was hard." "Yeah, but now that I'VE tried, we KNOW it's hard."
Read the whole story
Share this story
6 public comments
1240 days ago
Instead of algorithms it should be machine learning of course.
Seattle, WA
1240 days ago
Suggest having this strip at hand when reading Cathy O'Neil's book, Weapons of Math Destruction.
1240 days ago
For the last time, Joe, an algorithm can't explain why kids love the taste of Cinnamon Toast Crunch!
Moses Lake, WA
1240 days ago
My life in a nutshell
Austin, Texas
1240 days ago
"We TOLD you it was hard." "Yeah, but now that I'VE tried, we KNOW it's hard."
1240 days ago
ROFL. :)
Atlanta, GA

7 Liberal Lessons of Ebola

1 Share

Disease should make us think like a species, not like rugged individualists.

One perverse aspect of the public reaction to Ebola is the way it seems to be playing politically, at least in the short run. Ebola is making people afraid, and pushing them towards the party whose central narrative is about fear and anger: the Republicans.

Republican politicians are certainly playing up this angle: exaggerating the threat, and calling for xenophobic actions to combat it — cut off contact with Africa, seal the border against … well it’s not clear against who. Candidates have been amalgamating all the current fear-objects into one nightmare narrative: ISIS terrorists are going to infect themselves with Ebola, then sneak across our southern border to spread it here.

Senator Ron Johnson and Rep. Joe Wilson have put it most bluntly, but Republican Senate candidates around the country — Scott Brown, Thom Tillis, Cory Gardner — have been highlighting the pieces of this dark fantasy and hoping voters will assemble it for themselves: Ebola, ISIS, southern border.

Like most nightmares, this one evaporates as soon as you look at it by daylight: Ebola sucks as a bio-weapon, because it’s so hard to spread, and by the time the carriers were contagious they’d mostly just want to sleep; they certainly wouldn’t be able to swim the Rio Grande or hike the Arizona desert. Except in fantasy, no one has found any links between ISIS and Mexico. And unlike Texas, Mexico has no Ebola cases so far; if anybody should want to seal the border, it should be them, not us.

But nightmares — even very, very unlikely ones — raise fear, and fear makes people vote Republican. Or at least that’s what Republicans believe.

A rational person, though, ought to become more liberal when they think about Ebola, not more conservative. Here’s why.

1. Ebola points out why we need government. Libertarian rhetoric about sovereign individuals has a lot of superficial charm. But biology knows nothing about that; humanity is a species, and sometimes we have to act as a species. We do this through government.

If you want to get some distance on these issues, I recommend reading John Barry’s The Great Influenza, about the 1918-19 epidemic that killed as many as 100 million people around the world. The cities that did well with that plague were the ones whose governments were most draconian about it. As you read, try to imagine a plague hitting Galt’s Gulch, where each sovereign ubermensch would do his own research and make up his own mind about the disease and how to handle it. I don’t think they’d do very well.

There’s a lot of thankless, profitless work involved in controlling Ebola. For example, tracking down all the people who have been in contact with an infected person, and testing or quarantining them. It’s hard to imagine a free-market system that would do this well. The most obvious libertarian system would make individuals responsible for tracking their own exposure, and if some more complicated system created a profit motive for controlling a small outbreak, waiting until it’s a larger outbreak would be even more profitable.

2. Ebola points out why we need a fully funded government. When there’s no immediate threat of disease, government agencies like the CDC look like bureaucratic waste. When Rand Paul put out a “Tea Party budget” in 2011, it included a big cut in the CDC, and virtually no explanation as to how this would affect its mission. As I explained at the time:

sometimes you don’t get even that much justification, and the cut seems to be based on little more than an ideological assumption that waste must be in there somewhere. Take the CDC again. It’s our front line against plagues and epidemics, the folks we depend on to helicopter down in astronaut suits if SARS or ebola breaks out or drug-resistant tuberculosis gets out of hand. It has a total budget of $6.342 billion in 2011, so $1.165 billion represents a 28% cut for the final 2/3 of the year (assuming Paul’s bill could be passed immediately).

How should the CDC fulfill its mission with 28% less money? Given how disastrous a mistake could be, you might hope for some kind of expert justification, maybe a new strategy based on a medical study or two. Nope. The overview just suggests “focusing on domestic priorities rather than spending billions on overseas initiatives.” So basically, the CDC should stop worrying about plagues in other countries, and wait until they show up here. In Rand Paul’s world, that kind of thinking saves money.

I quote from my 2011 article to make this point: Hindsight wasn’t necessary to grasp how misguided this was.

NIH Director Francis Collins has speculated that we’d have an Ebola vaccine by now if not for the budget cuts that did get made: The $37 million we spent on Ebola vaccine research in 2010 was down to $18 million by 2014. Various other people have pushed back against that speculation. (And then Mike the Mad Biologist pushed forward again.) But the bottom line is simple: If you could reach back in time and reverse those cuts, wouldn’t you?

Now ask yourself: How many other cuts are like that? How many other agencies not currently in the headlines are we looking at as “wasteful spending” when it’s just that we don’t personally need them right now? And is it possible that events might make us wish we’d spent more before the emergency hit?

3. Ebola points out why we need a fully staffed government. Wouldn’t it be nice to have a surgeon general about now? (Just as it would have been nice to have had an ambassador to Russia when the Ukraine thing broke out or a Turkish ambassador as we were trying to get Turkey’s cooperation in opposing ISIS.) As former Surgeon General Regina Benjamin put it:

The surgeon general is America’s doctor. Delivering information to the American people in a language they can understand. Not having one right now, you don’t have that face and that person that the American people can identify with as their doctor who’s looking out for them on a large scale.

But we don’t have one because of the NRA. President Obama nominated Dr. Vivek Murthy back in March. But it turned out that Murthy views gun violence as a public health problem. (So does the AMA.) That makes him unacceptable to the NRA, so the Senate has been unable to confirm him (and a recent Supreme Court ruling prevents Obama from installing him as a recess appointment).

It wasn’t so long ago that the Senate believed in staffing the government, without making every appointment into a political football. But today’s Republicans have blocked Obama’s appointments on principle, even when they have no issue with the nominee. If they get control of the Senate in the upcoming election, expect the government to remain understaffed at least until the next administration.

If you’ve ever worked in an understaffed department, you know what that means: Stuff falls through the cracks. When that inevitably happens, Republicans will blame “government” rather than the true culprit: understaffed government.

4. Ebola demonstrates why we need to fund foreign aid. Foreign aid is one of the most unpopular parts of the federal budget (possibly because Americans grossly overestimate how much we spend on it). But viruses point out that the world is more interconnected than our political systems account for.

Bush administration officials used to tell us that we had to fight terrorism “over there” or else we’d eventually have to fight it over here. That’s debatable when it comes to terrorism, but it’s absolutely the fact when you talk about contagious diseases.

Ebola is controllable — previous outbreaks have been controlled, and the world has gone entire years without new cases. But ultimately it has to be controlled at the source, in west Africa.

Now widen your view a little: Anyplace in the world where people are living in unhealthy and unhygienic conditions, the next super-bug might be evolving. Any population that is “off the grid” of the global medical establishment might where a pandemic gets rolling before anyone notices.

5. The specter of a deadly infection demonstrates why we need universal health care. Conservative rhetoric revolves around individuals, and in particular how wrong it is to “give” individuals benefits — like health care — that they haven’t “earned”. Such individuals become “dependent on government” and take money away from “job creators”. It’s even worse if some of those benefits reach people who entered the country illegally or stayed past the expiration of their visas.

But when an infection gets loose, you want everybody who might be sick to seek treatment. You don’t want them to stay away from doctors because they can’t pay, or avoid the emergency room for fear of being deported, or not tell anybody about that undocumented cousin they might have infected.

I’m still not terribly worried about the spread of Ebola in the United States. (The number of cases and the likelihood of spreading the infection are both low.) But we might not be so lucky with the next disease. That’s why we should all be tremendously grateful that (so far) ObamaCare has gotten health insurance to ten million more people, and we should be working to plug the holes in that system rather than tear it down.

If a real epidemic got rolling, where would you rather be? In Massachusetts, where the model for ObamaCare, RomneyCare, became law in 2006, and only 1.2% of the population lacks health insurance? Or in a conservative wonderland like Texas, where 24.8% — probably including the Hispanics who clean your office or work in the kitchen at your favorite restaurant — are uninsured?

6. The Ebola panic demonstrates the danger of legitimizing conspiracy theories. During a plague, you need affected people to cooperate with the containment plan — seek treatment, accept quarantine, and report all their contacts truthfully — while unaffected people stay calm rather than doing panicky, stupid things. That’s when it’s important that the country trust its scientific establishment and its government.

Now of course it is important that the media and the political process police the trustworthiness of both those institutions. On those rare occasions when scientists fake data, they should be exposed. When the government lies, the media should investigate and seek the truth.

But what we’ve been seeing inside the conservative news bubble during these last six years goes way beyond that. Political opportunism has been seeking every opportunity to tear down public trust, even when — maybe especially when — the accusations are baseless.

And so, much of the public believes that the scientific community is involved in an elaborate conspiracy to promote a climate change “hoax”, or to destroy the Christian religion via the theory of evolution. So how can we believe what the doctors are telling us about Ebola?

And the Obama administration? If President Obama faked his birth certificate to hide the fact that he’s not really eligible to be president, if he’s been plotting to destroy the U.S. since he was a student, if he has a gun-confiscation plan that’s always just a month or two from implementation, if he is funding “death panels” that will decide whether your life is worth saving, if he has a “Kenyan, anti-colonial” worldview, if he “hates white people” or “has a deep-seated hatred of white people or the white culture” … why would his administration tell us the truth about Ebola? Fox News’ resident psychologist Keith Ablow lays it out:

[Obama's] affinities, his affiliations are with [Africans]. Not us. That’s what people seem unwilling to accept. He’s their leader … we don’t have a president. We don’t have a president who has the American people as his primary interest.

This is irresponsibility on a grand scale. Every era has a lunatic fringe with paranoid notions. But this kind of stuff comes from governors, members of Congress, a news network, and lots of other folks who seem to be part of a trustworthy establishment. And major national leaders — I’m looking at you, John Boehner and Mitch McConnell — sit at the same table and humor the purveyors of this destructive nonsense.

So it’s no wonder we’re seeing all kinds of weird behavior out there: Like the school in Maine that suspended a teacher for 21 days (the incubation period of Ebola) because she’s been to Dallas. Her hotel was less than ten miles from the hospital where two nurses got infected, so how can we have her in the same room with our children? (The local news report on this mentions a local parent who believes the government has “downplayed risk factors”. I wonder where he gets his news.) Thursday, several entire schools closed in Texas and Ohio because of Ebola contagion fears.

What would happen if we were having a real epidemic? I think mobs would be roaming the streets, burning down the houses of suspected carriers — all because the conservative movement and the Republican Party have prioritized destroying Obama over maintaining public trust in trustworthy institutions.

Pandering to people’s worst instincts may seem like a political freebie. But it isn’t. There’s a big social cost to this kind of stuff. But “social cost” is one of those things that conservatives are trained not to see. And that’s a 7th an 8th reason why you should be a liberal.

Read the whole story
Share this story

Google's Datacenters on Punch Cards

9 Comments and 25 Shares

Google's Datacenters on Punch Cards

If all digital data were stored on punch cards, how big would Google's data warehouse be?

James Zetlen

Google almost certainly has more data storage capacity than any other organization on Earth.

Google is very secretive about its operations, so it's hard to say for sure. There are only a handful of organizations who might plausibly potentially have more storage capacity or a larger server infrastructure. Here's my short list of the top contenders:

Honorable mentions:

  • Amazon (They're huge, but probably not as big as Google.)
  • Facebook (They're on the right scale and growing fast, but still playing catch-up.)
  • Microsoft (They have a million servers,[1]Data Center Knowledge: [Ballmer: Microsoft has 1 Million Servers although no one seems sure why.)

Let's take a closer look at Google's computing platform. try to figure out how much computing power Google has.

Follow the money

We'll start by following the money. Google's aggregate capital expenditures–spending on building stuff stuff—adds up to somewhere over $12 billion dollars. [2]I'm excluding the cost of an extremely expensive building they bought in New York. —adds up to somewhere over $12 billion dollars.[3]Data Center Knowledge: Google’s Data Center Building Boom Continues: $1.6 Billion Investment in 3 Months Their biggest data centers cost half a billion to a billion billion, dollars, so they can't have more than 20 or so of those.

On their website,[4]Data center locations Google acknowledges that they have datacenters in the following locations:

  1. Berkeley County, South Carolina
  2. Council Bluffs, Iowa
  3. Atlanta, Georgia
  4. Mayes County, Oklahoma
  5. Lenoir, North Carolina
  6. The Dalles, Oregon
  7. Hong Kong
  8. Singapore
  9. Taiwan
  10. Hamina, Finland
  11. St Ghislain, Belgium
  12. Dublin, Ireland
  13. Quilicura, Chile

In addition, they appear to operate a number of other large datacenters (sometimes through subsidiary corporations), including:

  1. Eemshaven, Netherlands
  2. Groningen, Netherlands
  3. Budapest, Hungary
  4. Wrocław, Poland
  5. Reston, Virginia
  6. Additional sites near Atlanta, Georgia

They also operate equipment at dozens to hundreds of smaller locations around the world.

Follow the power

To figure out how many servers Google is running, we can look at their electricity consumption. power bill. Unfortunately, we can't just sneak up to a datacenter and read the meter.[5]Actually, wait, can we? Somebody should try that. this. Instead, we have to do some digging.

The company disclosed that in 2010 they consumed an average of 258 megawatts of power.[6]Google used 2,259,998 MWh of electricity in 2010, which translates to an average of 258 megawatts. How many computers can they run with that?

We know that their datacenters are quite efficient, only spending 10-20% of their power on cooling and other overhead.[7]Google: Efficiency: How we do it To get an idea of how much power each server uses, we can look at their "container data center" concept from 2005. It's not clear whether they actually use these containers in practice—it may just have been a now-outdated experiment—but it gives an idea of what they consider(ed) reasonable power consumption. The answer: 215 watts per server.

Judging from that number, in 2010, they were operating around a million servers.

They've grown a lot since then. By the end of 2013, the total amount of money they've pumped into their datacenters will be three or four times what it was as of 2010. They've contracted to buy over three hundred megawatts of power at just three sites,[8]Google: Purchasing clean energy which is more than they used for all their operations in 2010.

Based on datacenter power usage and spending estimates, my guess would be that Google is currently running—or will soon be running—between 1.8 and 2.4 million servers.

But what do these "servers" actually represent? Google could be experimenting in all kinds of wild ways, running boards with 100 cores or 100 attached disks. If we assume that each server has a couple[9]Anywhere from 2 to 5 of 2 TB disks attached, we come up with close to 10 exabytes [10]As a refresher, the order is: kilo, mega, giga, tera, peta, exa, zetta, yotta. An exabyte is a million terabytes. of active storage attached to running clusters.

10 Exabytes

The commercial hard disk industry ships about 8 exabytes worth of drives annually. [12] [10] IDC: Worldwide External Disk Storage Systems Factory Revenue Declines for the Second Consecutive Quarter Those numbers don't necessarilly include companies like Google, but in any case, it seems likely that Google is a large piece of the global hard drive market.

To make things worse, given the huge number of drives they manage, Google has a hard drive die every few minutes.[11]Eduardo Pinheiro, Wolf-Dietrich Weber and Luiz Andre Barroso, [Failure Trends in a Large Disk Drive Population This isn't actually all that expensive a problem, in the grand scheme of things—they just get good at replacing drives—but it's weird to think that when a Googler runs a piece of code, they know that by the time it finishes executing, one of the machines it was running on will probably have suffered a drive failure.

Google tape storage

Of course, that only covers storage attached to running servers. What about "cold" storage? Who knows how much data Google—or anyone else—has stored in basement archives?

In a 2011 phone interview with Paul Mah of SMB Tech, Simon Anderson of Tandberg Data let slip [13] [11] SMB Tech: Is Tape Still Relevant for SMBs? that Google is the world's biggest single consumer of magnetic tape cartridges, purchasing 200,000 per year. Assuming they've stepped up their purchasing since then as they've expanded, this could add up to another few exabytes of tape archives.

All this could

Putting it all together

Let's assume Google has a storage capacity of 15 exabytes, [12]As a refresher, the order is: kilo, mega, giga, tera, peta, exa, zetta, yotta. An exabyte is a million terabytes. or 15,000,000,000,000,000,000 bytes.

A punch card can hold about 80 characters, and a box of cards holds 2000 cards:

15 exabytes of punch cards would be enough to cover my home region, New England, to a depth of about 4.5 kilometers. That's three times deeper than the ice sheets that covered the region during the last advance of the glaciers:

That seems like a lot.

However, it's nothing compared to the ridiculous claims by some news reports about the NSA datacenter in Utah.

NSA datacenter

The NSA is building a datacenter in Utah. Media reports claimed that it could hold up to a yottabyte of data, [14] [13] CNET: NSA to store yottabytes in Utah data centre which is patently absurd.

Later reports changed their minds, suggesting that the facility could only hold on the order of 3-12 exabytes. [15] [14] Forbes: Blueprints Of NSA's Ridiculously Expensive Data Center In Utah Suggest It Holds Less Info Than Thought We also know the facility uses about 65 megawatts of power, [16] [15] Salt-Lake City Tribune: NSA Bluffdale Center won’t gobble up Utah’s power supply which is about what a large Google datacenter consumes.

A few headlines, rather than going with one estimate or the other, announced that the facility could hold "between an exabyte and a yottabyte" of data [17] [16] Dailykos: Utah Data Center stores data between 1 exabyte and 1 yottabyte ... which is a little like saying "eyewitnesses report that the snake was "the escaped snake is believed to be between 1 millimeter and 1 kilometer long."

Uncovering further Google secrets

There are a lot of tricks for digging up information about Google's operations. Ironically, many of them involve using Google itself—from Googling for job postings in strange cities to using image search to find leaked cell camera photos of datacenter visits.

However, the best trick for locating secret Google facilities might be the one revealed by ex-Googler talentlessclown on reddit: [18] [17] reddit: Can r/Australia help find Google's Sydney data center? Seems like a bit of a mystery...

The easiest way to find manned Google data centres is to ask taxi drivers and pizza delivery people.

There's something pleasing poetic about that. Google has created what might be the most sophisticated information-gathering apparatus in the history of the Earth ... and the only people with the Earth ... yet the people with the most information about them are the pizza delivery drivers.

Who watches the watchers?

Apparently, Domino's.

Apparently, Domino's.

Read the whole story
Share this story
7 public comments
2557 days ago
There's a surprising amount of investigative work in this one.
Boston, MA
2562 days ago
ogrom googla
Warsaw, Poland
2562 days ago
Real life imitates Snow Crash. Again.
2562 days ago
So, essentially, you find Google the same way you find Torchwood?
2562 days ago
2555 days ago
Hee. "Illustration courtesy xkcd.com, used with permission."
2562 days ago
Hah! I know the guy that asked this one!
Austin, Texas
2562 days ago
http://what-if.xkcd.com/63/ Utah Data Center stores data between 1 exabyte and 1 yottabyte ... which is a little like saying "eyewitnesses report that the snake was between 1 millimeter and 1 kilometer long."
North of Boston