Menu

06/18/15 Frederic Lardinois – Techcrunch Knowledge

Google Pulls Back Curtain On Its Data Center Networking Setup


Google

While companies like Facebook have been relatively open about their data center networking infrastructure, Google has generally kept pretty quiet about how it connects the thousands of servers inside its data centers to each other (with a few exceptions). Today, however, the company revealed a bit more about the technology that lets its servers talk to each other.

It’s no secret that Google often builds its own custom hardware for its data centers, but what’s probably less known is that Google uses custom networking protocols that have been tweaked for use in its data centers instead of relying on standard Internet protocols to power its networks.

Google says its current ‘Jupiter’ networking setup — which represents the fifth generation of the company’s efforts in this area — offers 100x the capacity of its first in-house data center network. The current generation delivers 1 Petabit per second of bisection bandwidth (that is, the bandwidth between two parts of the network). That’s enough to allow 100,000 servers to talk to each other at 10GB/s each.

Google’s technical lead for networking, Amin Vahdat, notes that the overall network control stack “has more in common with Google’s distributed computing architectures than traditional router-centric Internet protocols.”

Here is how he describes the three key principles behind the design of Google’s data center networks:

  • We arrange our network around a Clos topology, a network configuration where a collection of smaller (cheaper) switches are arranged to provide the properties of a much larger logical switch.
  • We use a centralized software control stack to manage thousands of switches within the data center, making them effectively act as one large fabric.
  • We build our own software and hardware using silicon from vendors, relying less on standard Internet protocols and more on custom protocols tailored to the data center.

Sadly, there isn’t all that much detail here — especially compared to some of the information Facebook has shared in the past. Hopefully Google will release a bit more in the months to come. It would be especially interesting to see how its own networking protocols work and hopefully the company will publish a paper or two about this at some point.


Related Tags

Google

All Research Tags

$DFT $DRT 2021 Data Center acquisitions 2024 Data Centers 2103 data center acquisition report 350 E Cermak AI AI Data Center AI Data Center Boom AI Data Center Consulting AI Data Center Growth AI Data Center Site Selection AI Data Centers AI Growth AI M&A AI Real Estate AI nuclear power data centers AT&T AWS AWS Cloud AWS Columbus AWS Data Centers AWS data center Amazon Cloud Amazon data center Artificial Intelligence Data Centers Ashburn Data Center Market Atlanta Data Center Leasing Atlanta Data Center Market Azure BC Partners buying Centurylink data centers Barron's Data Center Real Estate Bitcoin Mining North Carolina Bitcoin Mining Power Rates Bitcoin Mining Washington Blockchain Business Continuity Buys Data Center Buys Data Center 2018 Buys Data Center 2019 Buys Data Center 2020 C7 Data Center CIO CIO Insight CIOForum CONE CONE REIT Sale CONE Sale COR CTO Carter Validus REIT Carter Validus sells data centers Centurylink Data Centers Centurylink buys Level 3 Centurylink data center business Centurylink selling data centers Chanos Cloud Cloud Computing Cloud Data Centers Cloud Spending Colocation Columbus Data Center Market CoreSite Coresite Covid-19 Cryptocurrency Mining Power Cryptocurrency mining space CyrusOne CyrusOne Acquisition CyrusOne Atlanta data center CyrusOne Bidder CyrusOne Data Center Atlanta Cyxtera Technologies DCRE DLR DLR buys DFT DRT Dallas Data Center Activity Dallas data center Data Center Data Center AI Data Center Absorption Data Center Absorption 2021 Data Center Acquisitions Data Center Acquisitions 2019 Data Center Acquisitions Report 2018 Data Center Activity 2018 Data Center Broker Data Center Brokerage Data Center Cloud Demand Data Center Consolidation Data Center Construction Columbus Data Center Consulting Data Center Covid Data Center Development Data Center Growth Data Center Investment Data Center Investments Data Center Investments 2020 Data Center Land Data Center Leasing Data Center M&A Data Center Market Data Center Market 2021 Data Center Market Research Data Center Mergers & Acquisitions Data Center Multiple Data Center News Data Center Power Data Center Purchase Data Center REITs Data Center Real Estate Data Center Real Estate Investing Data Center Rent Data Center Report Data Center Research Data Center Tax Incentives Georgia Data Center investing Data Center leasing report Data Centers Data Centers 2019 Data Centers 2020 Data Centers For Sale Data center absorption 2018 Data centers DataBank DataCenters# Databank DevOps Digital Bridge Digital Realty Digital Realty Trust Digital Realty Trust Atlanta Digital Realty Trust Chicago Digital Realty Trust Dallas Digital Realty Trust acquires Dupont Fabros Digital Realty Trust merger Digital Realty buys Chicago data center Digital Realty buys Dupont Fabros. Dupont Fabros Dupont Fabros merger Durham Data Center EQIX Enterprise Data Centers Equinix Equinix Dallas Equinix buys Infomart Equinix buys Verizon Data Centers F5 Networks Facebook Atlanta Data Center Facebook Data Center Ft. Worth Data Center GI Partners GOOG GOOGL Gartner Georgia Data Center Legislation Global Switch Google Google Cloud Google Data Centers HPC Data Centers High Performance Computing Data Centers Hyperscale Hyperscale Data Centers Hyperscale data center locations Hyperscale data centers Hyperscale growth IBM IBM data centers IPFS IRM data center expansion Infomart Infomart Dallas Iron Mountain Iron Mountain Data Centers Iron Mountain buys Fortrust Iron Mountain buys IO Data Centers KKR Level 3 MSFT MapleTree buys data centers Meta Columbus Microsoft Microsoft Cloud Microsoft Data Center Microsoft data center NOVA Data Center market NOVA Hyperscale Data Centers NXTVn New Chicago Data Center New Jersey Data Center Northern Virginia Absorption Northern Virginia Hyperscale Data Centers Nuclear power data centers Nvidia Nvidia AI Data Centers Oracle Cloud Oracle data center Peak 10 Private Equity Data Centers Private equity data centers Public Cloud QTS QTS Data Center Atlanta QTS data center Redhat data center cloud Santa Clara Data Center Secondary Data Center Markets Sells Data Center Sells Data Center 2019 Sentinel Shaw Communications Switch Switch Data Center Atlanta Switch data center Atlanta T5 Data Center Atlanta Telefonica TierPoint TierPoint# Time Dotcom US Low Cost Power States US Power Grab US data center market report 2018 Vantage Vantage Data Centers Verizon ViaWest Virginia Beach Cable Landing Station Virginia Beach Data Center Park Virginia data center park Wall Street Journal Windstream Zayo ata center blockchain big data blockchain data center blockchain data storage blockchain file storage buys data center and colocation business buys data centers cloud data center cloud data centers data center data center acquisitions data center advisory data center broker data center consultant data center consulting data center for lease data center for sale data center investments data center leasing data center market data center market research data center news data center outage data center real estate data center sale leaseback data center sales data center tax incentives data center trends data centers data centers for sale delta data center delta outage dropbox data center emerging data center trends hybrid cloud hyperscale data centers north carolina data centers renewable nuclear energy data centers