What should a "Space Force" look like?

For roughly the last 35 years, since the start of the Space Shuttle program in 1981 with its planned dual use launching both civilian military payloads, the latter including military-only crews from Vandenberg AFB's Space Launch Complex 6 (SLC-6, or "Slick Six"), and the establishment of Air Force Space Command in 1982 and later U.S. Space Command in 1985, there has been talk in the military and political circles about an independent "Space Force."

"Space Force" talk peaked at the end of the last century after the Clinton Administration and Congress chartered the Commission to Assess United States National Security Space Management and Organization in 1999. The commission issued its report in January of 2001, and not much happened.

The commission proposed several suggestions, including the idea of an independent Space Corps within the Department of the Air Force. At that time, Air Force Space Command (AFSPC) included the 20th Air Force and the USAF's ICBM mission. Since that time, Air Force Global Strike Command (AFGSC) was created to manage the nuclear deterrence mission and 20th AF passed to AFGSC. AFSPC took ownership of the USAF's Cyber mission and 24th AF, but this will be passed to Air Combat Command in the near future.

The move of 24th AF is likely due in part to congressional discussions about creating a sub-unified U.S. Space Command out of U.S. Strategic Command's Joint Force Space Component.

When one looks at the history of military space functions in the various military branches over the last 40 years, one sees the following areas of military space efforts.

The U.S. Navy gave over some of its space missions to the USAF. The Navy's Space Surveillance System (the "Space Fence") moved to Air Force control in 2004 and subsequently was shuttered in 2013. The Navy managed communications satellite programs remain with the Navy, as does the Navy's role in ballistic missile defense (BMD).

The U.S. Marine Corps has no significant space mission area, and only a skeleton staff at USSTRATCOM.

The U.S. Army's primary space related role is ballistic missile defense, which is significant. The Army contains the second largest military space organization in its Space and Missile Defense Command.

What mission areas should a Space Corps encompass? Should it include the National Reconnaissance Office (NRO)? Should it include ballistic missile defense? Should it include the ICBM deterrent force? All of these are subjects for debate, but from these we can assume a reasonable maximum scope--all of the above, less the Navy's operational BMD role, host base responsibility for Kirtland Air Force Base (currently under 20th AF), and the 582d Helicopter Group (under 20th AF). The Navy's Aegis Ballistic Missile Defense System is simply a capability of the Aegis Weapon System and cannot be divided from its fleet defense role.

Who would lead such an organization? The obvious choice would be the current 4-star Air Force Space Command commander, who would become the commandant of the Space Corps. The deputy could logically be the current 3-star Army Space and Missile Defense Command commander, but promoting him or her to a 4-star grade equal to other service deputies. Of course both of these leaders would move to the Pentagon where the new service would be headquartered.

That creates the following as a potential U.S. Space Corps organization:

 

Air Force and Army personnel associated with the new Space Corps would be assigned, with the former Army personnel ultimately transferring to the U.S. Air Force after a transition period.

A period of transition would allow moving Army Space Cadre and Army Space Operations Functional Area personnel, and Air Force space and missile primary AFSC personnel, who currently are assigned outside of the prospective Space Corps units to move into the Space Corps.

Likewise, those who are serving in space and missile roles who do not maintain a space or missile primary MOS/AFSC would be given time to return to assignments associated with their primary MOS/AFSC.

A challenge for Army personnel is this connection of modern BMD personnel to historical Air Defense Artillery Branch. The 100th Missile Defense Brigade, the operational BMD unit of the Army Space and Missile Defense Command no doubt has personnel who shift between the national missile defense roles and theater Air & Missile Defense and Air Defense Artillery organizations. This is why the majority of the personnel who move to the Space Corps via inter-service transfer need to be from the Army Space Cadre. Certainly some exchange postings could be made available, more initially, fewer later, to ease the separation of theater and national missile defense roles.

The small number of uniformed Navy personnel associated with the new Space Corps should be given the option of transferring to non-Space Corps related positions on their next move, but those with space related specialties should be allowed to transfer to the Space Corps.

Finally, decisions would need to be made regarding supporting MOS/AFSC positions as to who would transfer to the Space Corps, and who would remain with their parent services. Most of these are traditional host installation support roles, such as a civil engineering and security, as well as traditional personnel and administration positions. As an independent service within the Department of the Air Force, at least initially the host installation support positions could remain with the Air Force.

The initial total end strength of the Space Corps will be 20,000 to 25,000 (depending on the level of host installation support personnel transferred), and will likely have 30-35 general officers leading the various headquarters.

Some host installation support and medical support should remain within the Air Force to minimize unnecessary duplication. Officer accession and senior officer professional military education should continue to be a part of the Air Force, while enlisted professional education and mid-grade officer professional education (Command and Staff College) should be tailored to the needs of the Space Corps. Enlisted accession could be co-located with Air Force enlisted training but operate separately.

This is most interesting. And it is not the half of it.

This is most interesting. And it is not the half of it.

Updated November 27, 2025.

Why We are Alone in the Galaxy | Marc Defant | TEDxUSF

[EDIT: Added 12/19/24]

I noticed TED has "flagged" Dr. Defant's video, apparently due to questions about some of his scientific claims. I found one serious error in the chart of the log of brain mass to the log of body mass. But I can find no other errors in what he speaks to. But is everything else, his science is extremely sound. Here are links to the three topics he speaks to:

The creation of the solar system, and dating due to calcium–aluminium-rich inclusions in the Allende meteorite:

Allende meteorite

Calcium–aluminium-rich inclusion

Aluminum-26 production from a stellar evolutionary sequence

That said, Dr. Defant is incorrect when he says "all" of the elements in the periodic table beyond hydrogen and helium were generated by supernovas. Some scientists believe lithium was created prior to star formation, and most scientists believe medium weight elements up to iron and nickel were created in stars. The process that creates the elements from beryllium to nickel is called stellar nucleosynthesis. This is also called the "slow process" or the "s-process." However, some scientists believe the initial phase of a supernova (the collapse of the star) creates conditions to fuse iron with other elements to create some of the elements heavier than iron. This is called the "rapid process" or the "r-process." The supernova explosion then releases the elements into the void of space.

Stellar nucleosynthesis

The creation of many elements including iron and copper are heavily dependent on Type Ia supernovas, a supernova of a white dwarf star which is part of a binary star system. Other elements are dependent on dying low-mass stars, which become red giants, then dissipate into planetary nebulas. Many scientists now believe very heavy elements like platinum, gold and heavier could only be generated in a neutron star. That a supernova collapse is not powerful enough to cause the r-process, and instead the intense gravity of a neutron star is able to do so. However, given the high gravity of a neutron star, the only way these elements could be released is by the collision of two neutron stars (a kilonova). This most likely occur in a binary neutron star formed for the collapse of a binary star system. The possibility of a single wandering neutron star colliding with a single nearby neutron star, given the average diameter of a neutron star is about 12 miles, seems an almost infinitesimal probability. However, neutron stars are the remnants of a core of a star that went supernova, so perhaps Dr. Defant consolidated these topics for the sake of time.

The following article talks about the r-process:

How are heavy elements formed in neutron star collisions?

Regardless, if the medium weight elements were generated in stars and release via supernovas, and there is no consensus on the supernova collapse creating heavier elements, we also needed nearby dying low-mass stars, a nearby white dwarf that was part of a binary star system, and a nearby binary neutron star systems that collided to give our solar system the elements from copper to uranium. We know a kilonova seeded our stellar nebula, because we have very heavy elements from silver to uranium in the Earth's crust. When you look at the abundance of elements in Earth's crust, while iron is plentiful, once you move above iron in atomic weight, the rest of these heavier elements make up less than one-half of one percent of the elements in the Earth's crust.

Visualizing the Abundance of Elements in the Earth's Crust

(Source: Visual Capitalist)

Copper and tin are incredibly important elements. They are the key elemental metals used to form the alloy bronze, and as such were key to the bronze age. Copper is created from the s-process or Type Ia supernovas. Tin is created either by a dying low-mass star or the r-process. Bronze was the basis for hard metal tools like nails, spearheads, plows, etc. The problem is copper and tin are rare compared to iron. Iron is one thousand times more prevalent in the Earth's crust than copper. Iron is more than 25,000 times more prevalent than tin, antimony, or arsenic. It makes sense to believe there may be planets which have iron, but have no tin, antimony, or arsenic, meaning no ability to make bronze. But iron requires considerably more heat to shape and purify (through heating and hammering). The earliest bronze dates to 4,650 BCE, while the earliest shaped iron dates to about 2,000 BCE.  The historical "Bronze Age" is pegged to 3,300 BCE to 1,200 BCE, which gave way to the "Iron Age" in 1,200 BCE. Regardless of how you measure it, for about 2,000 critical years, the only metallurgy that existed in human society was done with copper and tin.

It seems highly possible there could be planetary systems created with only the medium weight stellar elements, and few heavier elements. But if there was no tin, antimony, or arsenic, any intelligent life would have to go from a stone age to an iron age, and have to figure out how to smelt iron. Without iron, anything requiring hardness would need to use stone or pottery. Copper alone is too soft. There would be no opportunity to create metal nails or spikes until iron was mastered. It is possible an intelligent society would develop enough skills in an extended pottery age to transition to an iron age, but this seems less likely, because the uses of pottery and iron are very different, unlike the transition from bronze to iron. There is a significant chance an intelligent society without tin, antimony, or arsenic, and the opportunity for a bronze age would never perfect iron smelting, and instead remain stuck in a stone age.

But back to the not one, but four cosmic events, a nearby supernova of a massive star, a nearby low-mass star death, a nearby Type Ia supernova of white dwarf in a binary star system, and a nearby collision of two neutron stars (which could be remnants of the same nearby massive star supernovas that contributed the lighter elements), to Dr. Defant's original point, that would be incredibly rare. Even if there as a nearby binary star system, with nearly simultaneous supernovas, each leaving behind a neutron star forming a binary neutron star, that then merged and released heavier elements, while this is an easier explanation, it is also rare. And it still likely needs a dying low-mass star and a nearby Type Ia supernova. All of this is far more rare than just a nearby supernova.

The next point Dr. Defant makes is the rise of the mammals being a direct result of the extinction of the dinosaurs:

The Rise of Mammals

And then there is the East African Rift directly triggering the evolution of bipedalism in great apes and rise of pre-humans:

Did tectonic rift push apes and monkeys apart?

How a changing landscape and climate shaped early humans

[EDIT: Added 12/19/24]

Regarding the chart error, the chart Dr. Defant uses already includes the human log of brain mass to log of body mass. Dr. Defant has a 10X error which puts his red dot well to the left of where it should be. However, even if it were correctly positioned, it would be an outlier on this chart.

Here is a screen capture with an arrow pointing to the actual dot representing homo sapiens:

Yes, there will be some mammals with larger body masses and larger brain masses, but among the primates, Humans are significant in both their absolute brain mass and their brain mass to body mass ratios. Further more, from an evolutionary perspective, human brain growth is unusual. Over a period of 2.4 million years, the human brain went through two 600,000 year periods of rapid growth, each time roughly doubling in mass, along with two 600,000 year periods of minimal growth, resulting in humans having relatively gigantic brains compared to other hominids. This is something that is hard for evolutionary biologists to explain, because a doubling in brain size in 600,000 years is not expected under neo-Darwinian evolution timelines.

[EDIT: Added 6/28/25]

Dr. Defant's use of brain mass to body mass has some merit, but I discovered another metric which is more useful, the Encephalization Quotient, or or just encephalization. Encephalization is a relative brain size measure that is defined as the ratio between observed and predicted brain mass for an animal of a given size. It has been used as a proxy for intelligence and thus as a possible way of comparing the intelligence levels of different species. It takes into account allometric effects. (Allometry is the study of the relationship of body size to shape, anatomy, physiology and behavior.) The result is to show how many "extra neurons" a particular animal has above what they need based on a biological study of the animal. This extra brain mass is assumed to be available for higher cognitive function. 

Humans have an encephalization quotient of 7.4 to 7.8 compared to a chimpanzee's encephalization quotient of 2.2 to 2.5. That puts humans at 3 to 3.5 times the "extra brain" capacity of our nearest primate relatives. This compares to 2.4 times the brain to body mass ratio. Humans are extreme outliers among primates.

But wait, there's more!

Where is Everybody? Why Haven't We Found Extra-Terrestrials?

A little more on this. The Earth and the Moon are a system. The Moon does not neatly orbit around the center of mass of the Earth. Instead, the center of mass of the Earth, and the center of mass of the Moon, both orbit around the center of mass of the pair (the barycenter). This wobbling makes it much harder for an object being pulled towards the center of mass of the Earth-Moon system to follow a straight line. All of that extra mass out there is pulling an object away from the center of mass of the Earth-Moon system.

SkyMarvels™ EARTH-MOON BARYCENTER (celestia celestia4all)

As an example of how this works, look at the crazy, almost half-century trip of Apollo 12's third stage. It was fired out into a heliocentric orbit around the Sun, but kept getting pullled back into the Earth-Moon system's gravity well into a geocentric orbit around us, then ejected out into a heliocentric orbit again, and then back into geocentric orbit. It is like a piece of driftwood bobbing along a tideline in the ocean, sometimes getting pulled away by one current, sometimes by the other.

J002E3 - Wikipedia

[Edit: Added 12/19/24]

Regarding Bill Whittle's comments on the total solar eclipse. Below I added a link to the documentary film "The Privileged Planet." in it Dr. Guillermo Gonzalez speaks to the concepts of a G-type main-sequence star being ideal for a solar system that supports the development of life, the circumstellar habitable zone around such a star, the mass of a planet that could support the atmosphere and magnetic field requirements for life, the benefits of an axial tilt providing seasons, and the benefit of a single, large satellite which would maintain the axial tilt and also provide ocean tidal forces, and he notes such a situation would set up a higher potential for a total solar eclipse. Still rare the Moon perfectly obscures the Sun, but then Gonzalez points out that allowed observations of the solar corona in the 19th and early 20th century that led to major advancements in stellar astronomy. He makes one error in that he says a larger satellite would have prevented this. A larger satellite would provide a brief opportunity to observe the solar corona before it was obscured. However a smaller satellite would at best offer an annular eclipse, which would be too bright to observe the solar corona.

But wait, there is even more!

Jupiter shepherds large transient objects (asteroids) into a belt outside the orbit of Mars. The very large mass of Jupiter, well outside of the inner Solar System, is a unique and very good thing for us fragile creatures on the Earth. Notice the Asteroid Belt is not a neat circle in between the orbits of Mars and Jupiter, it is very different. That will be explained in the post that follows.

The asteroids of the inner Solar System and Jupiter

Jupiter 'shepherds' the asteroid belt, preventing the asteroids from falling into the sun or accreting into a new planet.

Now, the above video is beyond cool, because it illustrates four of the five Lagrange points (L1, L3, L4, and L5).

Lagrange Points: Parking Places in Space

In addition to shepherding space objects, Jupiter "Hoovers" these objects just like Earth's Moon.

Here Joe Scott of "Answers With Joe" explains the "Rare Earth Theory", which includes the role of Jupiter at the 9:09 mark:

Joe's video goes much further than this blog post on the habitability of planets within the Milky Way galaxy, and the role of Earth's magnetic field. Check it out.

Basically, the "Habitable Zone" is protected by Jupiter, we have Earth at an ideal distance, with the right chemical composition, and we have our own Moon as a terminal defense system.

[EDIT: Added 12/19/24]

Here is a link to the 2004, one hour long documentary based on the 2004 book "The Privileged Planet" by astronomer Guillermo Gonzalez and philosopher Jay Richards. It focuses on Rare Earth theory with a secondary focus on Intelligent Design. Even if you do not accept the concept of Intelligent Design, the Rare Earth aspects of the documentary are very good. This book and documentary were made when only a few hundred exoplanets had been discovered, and details were limited. The fact they hit on the exact points that were later exposed (see below) with more recent exoplanet discoveries, and did this 20 years ago, means their book and film have stood the test of time.

There are two additional important considerations.

NASA has discovered many exoplanets. Some of these orbit cool stars, such as "Red Dwarf" within a "habitable zone" which is much closer to the star. This arrangement can lead to tidal locking, which is problematic.

 

Tidal locking

Tidal Locking Could Render Habitable Planets Inhospitable

Another consideration if a planet is close to a cool star is the planet could be exposed to high high levels of radiation. Both tidal locking and high radiation are factors in the TRAPPIST-1 plantary system discovered in 2015.

TRAPPIST-1

The topic of radiation brings up the final point of a planet capable of supporting life. Of the solar system's three inner planets in or near the "habitable zone" (Venus, Earth, and Mars), only Earth has a magnetic field. Earth's magnetic field deflects harmful high-energy cosmic rays coming from the Sun away from the Earth. Mercury has a magnetic field, but is too close to the Sun to be habitable. Jupiter has a magnetic field, but as a gas giant is not considered habitable.

[EDIT: Added 12/19/24]

I have not gone into circumstellar habitable zones, or galactic habitable zones. Dr. Guillermo Gonzalez, the coauthor of the book "The Privileged Planet" mentioned above was one of the key researchers who contributed to the galactic habitable zone concept. The film goes into more detail on both circumstellar and galactic habitable zones.

[Edit: Added 11/26/19]

Most of this post has dealt with what it takes to form a stable solar/planetary system capable of supporting life. However, life is complicated, and no scientist has figured out abiogenesis (life from lifelessness). But even with that spark of abiogenesis, how does complicated and sophisticated live emerge? The assumption has been Darwinian Evolution. Indeed Darwin's theory is the nexus of what today is called "neo-Darwinism." However, some believe neo-Darwinism has breached the limits of mathematics. This excellent discussion with David Berlinski, Stephen Meyer, and David Gelernter, which was triggered by David Gelernter's article "Giving Up Darwin", is worth a listen:

I often like to joke if life were easy, we would all have green thumbs. Anyone who has had a tried to keep a house plant or a goldfish alive should realize even in a stable biosphere, life is fragile.

And all of this first presumes a stable universe, where all of the key physical forces are balanced so that atoms form and matter condenses into stars and planets.

I am not suggesting any conclusion from all of these comments and videos except one: Life is incredibly rare and precious. Far rarer than many believe. It is certainly possible someday we will find evidence of primitive life on Mars, but Mars was unsustainable. It is certainly possible someday we could terraform Mars, but the terraformed Mars will not last. Because its gravity is low and it has no magnetic field, its atmosphere will dissipate, and in a hundred, thousand, or ten thousand years, it will be back to what it is today. Besides, if we ever have the technology to terraform a planet, we will be able to fix any problems on Earth.

[Edit: Added 12/22/22]

Having further researched some of the early cosmological events, I now think the Earth, with a very small amount of critical heavier elements which could only be manufactured in a neutron star, and released by a collision with another neutron star, and the critically important nature of copper and tin in metallurgy and the rise of civilization, is even more rare than I first thought. It is very likely there are planetary systems with no elements above nickel in their crusts. There may be intelligent life on these planets, but they would likely be trapped in a stone age forever. On the bright side, they would never develop nuclear weapons, and likely not even develop cannons or firearms, so they could have less chance of destroying themselves, but it would also be harder for them to feed themselves as they never would have bronze plows. Certainly, without copper, they likely would never be able to build a radio transmitter to send a signal SETI could pick up.

[Edit: Added 3/22/24]

I updated several links due to some dead links and some URLs changing. No content was changed.

[Edit: Added 4/10/24]

I added an additional depiction of the asteroid belt to show the main belt as well as those shepherded by Jupiter.

[Edit: Added 12/19/24]

I added some commentary describing an error made by Dr. Defant, and included a video link to the 2004 documentary "The Privileged Planet", which covers Rare Earth theory and described some of the expected issues with exoplanets 20 years ago.

[Edit: Added 6/28/25]

I added a definition of encephalization quotient and a link to its Wikipedia page, along with some commentary.

[Edit: Added 11/27/25]

Minor updates and edits.

Thoughts on HyperConverged, and the Future of HyperConverged (Part 2)

So how did we get here? Where did HCI come from?

If we look back at the history of HCI, it seems to have evolved from the idea of using clustered, "whitebox" x86 servers to create a clustered storage system. There were a number of early entrants in the space, some dating back to 2006. Another vector was the idea of a "Virtual Storage Appliance" or VSA, software which ran in a VM, connected to local server hard disk drives, and presented that internal storage to the guest VMs over the internal IP network. The first VSA was from Lefthand in 2007. But the real hyper-converged push started around 2009 with the founding of integrated HCI players Nutanix and SimpliVity.

We also have to look at where the HCI market is today. It is arguably dominated by three primary players: Nutanix; SimpliVity (now part of HPE); and VMware VSAN. They represent the lion's share of the HCI market, and we will come back to them.

If you look at the earlier clustered storage companies, they either offered a scale-out NAS, a kind of commodity alternative to Isilon, a scale-out block storage solution, or a scale-out unified storage solution. These early players came into existence when "grid computing" was the buzzterm of the day, and these architectures were also called "grid storage".

In 2009 Nutanix was founded. There were other virtual storage appliance start-ups, such as Virsto Software (which eventually became VMware VSAN), but it is fair to define the official beginning of the hyper-converged era as August 2011, when Nutanix emerged from stealth. The same month, VMware released vSphere 5 which included its first implementation of a VSA (vSphere Storage Appliance). SimpliVity would emerge from stealth one year later in August 2012. VMware's VSA did not gain traction, and VMware announced its intent to acquire Virsto six months later in February 2013 which represented VMware's serious interest in HCI.

As Nutanix and SimpliVity started to grow, and with VMware's very public acquisition of Virsto, and obvious plans to enter the HCI market, many of the earlier clustered storage vendors and virtual storage appliance vendors redefined themselves as hyper-converged players. Several new industry buzzterms were developed: "Server SAN"; "Virtual SAN"; and "Software Defined Storage", or "SDS".

Many of the early clustered storage system vendors redefined themselves as SDS or HCI players, moving their clustered storage software from bare-metal to run in VMs, and allowing their clustered storage software to run alongside guest VMs on the same server. VSA vendors added more sophisticated clustering, replication, and scalability to their products.

From this, it is fair to say modern HCI owes itself to three parents: Commodity clustered storage systems; virtual storage appliances; and purpose built integrated HCI systems.

To me, the most interesting thing is many of the earlier clustered storage or "grid storage" players had little to no success, but the HCI players saw significant early success. Part of this may have been how each targeted the market. Clustered/grid storage historically had been seen as targeting the high-performance and academic community for technical computing use cases. HCI targeted business organizations and VMware virtualization workloads.

But what cannot be dismissed is the reality the early clustered storage ystems did not provide the level of performance and reliability required for enterprise workloads. The early clustered storage systems were not designed for transactional, random I/O workloads. They were better suited for sequential I/O. The early HCI players focused on addressing write latency and random I/O with aggressive write and read caching. The also focused on ease of use and eliminating the need for storage administrators to provision storage to VMware administrators.

At this point it is interesting to note, there were other players aggressively targeting VMware virtualized workloads. Tintri had come out of stealth five months before Nutanix with its VMware optimized storage platform. It too targeted the VMware admin and sought to used its product to bypass the traditional storage management team in an organization.


So that is the history lesson and the end of Part 2.

Thoughts on HyperConverged, and the Future of HyperConverged (Part 1)

Almost two years ago I made some observations on HyperConverged Infrastructure, and where I think it needed to go to be successful. I posted these to Twitter at the time. I still stand by some of those observations, for others I am not as sure. But I have done a lot more thinking about the HCI phenomena, and believe change is coming to HCI.

To this point, I recently saw an update of the Gartner Hype Cycle, which showed HCI at the zenith of the "Peak of Inflated Expectations". I agree with this. The question is what comes next? Probably a vendor shake-out.

 
But another question to ask is "What comes after HCI?" The idea HCI is the end-game for IT infrastructure is a naive assumption. There may be better architectures being worked on by start-ups as I write this.

These were my original observations on HCI:

HCI must support multiple hypervisors, and no hypervisor (i.e., Containers, Hadoop, Oracle RAC, etc.).

At the time, Microsoft was pushing Hyper-V very hard, and I thought Hyper-V was going to make significant penetration into the enterprise. At the same time, some organizations were experimenting with OpenStack and KVM. Today, looking back, VMware still dominates. Hyper-V exists mainly in on-prem Azure Stack deployments, and KVM struggles without a single brand behind it.

As for no-hypervisor HCI (my idea being a combination of OpenStack with Containers and an HCI filesytem embedded in Linux for something like Oracle RAC), this has yet to take off. There is a chance we could see something like it for OpenStack.

HCI must become all-flash for virtualized workloads.

For the most part, this has become true. And the reality is, All-Flash saved HCI, which probably would not have been able to keep up with the performance requirements of virtualized workloads in its hybrid form.

HCI filesystems must be or become flash aware (WAF, etc.).

HCI filesystems have been adapted for flash, but I do not believe they have reached a point to make them comparable to All Flash Arrays in reducing flash wear. They have been able to avoid this by using high Drive Write Per Day (DWPD) SSDs in their caching tier to coalesce writes to low DWPD SSDs in their capacity tier. I see two problems with this approach. The first is the use of a high DWPD SSD as a cache is a carry-over from the hybrid HCI filesystem architecture. There it provided a significant performance boost. When combined with an SSD capacity tier, it provides no performance boost, and only a write wear mitigation benefit. The second issue is high DWPD SSDs are not a high volume part for SSD manufacturers, who would rather manufacture lower DWPD, higher capacity, higher revenue SSDs. Ultimately, high DWPD SSDs may fade away like SLC and eMLC SSDs did. If that happens, what will HCI vendors do?

HCI must move to parity/erasure coding data protection and move away from mirroring/replication based data protection (RF2/RF3).

I believed this was necessary for All-Flash HCI due to the cost of flash, and the capacity of SSDs at the time. I am less sure of this now, at least as a $/GB requirement. I think parity/erasure coding will only be driven by availability requirements, and not $/GB requirements.

HCI must support storage only nodes and compute only nodes for asymmetric scaling.

I believe this even more today. With All-Flash HCI, storage efficiencies (a.k.a., Data Reduction technologies) became critical. When you look at the Virtual Desktop (VDI) use case for HCI, deduplication means storage capacity does not grow linearly with VDI instances. In fact, it hardly grows at all. But what does grow is a need for write caching. If I invested in HCI for VDI, and deployed 200 VDI instances across 4 HCI nodes, and later decided to grow my VDI to 400 instances, I might need 4 more nodes of compute, but deduplication might mean I need only 10% more storage capacity, which I might already have on my existing nodes. I might need a caching SSD on each new node, but not 5 to 11 data drives.

The reverse holds true as well. If I assume a certain storage efficiency ratio, but due to adding workloads with different data types (say pre-compressed image files) my storage efficiency drops, today I have to add compute and hypervisor instances (and associated licenses) just to gain access to more storage capacity. If I could add a storage only node or two, it would provide flexibility. Also, it might offer the ability to introduce tiering between an all-SSD production tier, and a NL-SAS capacity tier.


This is the end of Part 1. Over the next several parts, I will dig much deeper into these thoughts, including thoughts on what comes after HCI.

The Big Payoff

The big payoff for driverless vehicles is with driverless trucks, not driverless cars, especially driverless "Ubers". By driverless trucks, I specifically mean long-haul trucks.

 

A typical long-haul trucker drives 10 hours a day, meaning the truck is idle the other 14. Some downtime is needed for refueling, weigh stations, etc., but it is reasonable that driverless long-haul trucks will double the productivity of human driven trucks, quite literally overnight.

There has been a shortage of people willing to work as long haul truckers, even given it pays a middle-class income without the need for excessive education or training. This has caused labor costs to rise.

There are currently over 1.5 million long-haul truckers and estimates are the need for long-haul truckers will approach 2 million in the next 5 years.

There are about 250,000 taxi and limo drivers, and they make less than long-haul truckers. Uber and Lyft have exposed there is much greater demand for car services than originally expected, and the capital-less model of ride-shares works well for that demand. The flood of ride-shares has depressed wages for both taxis and ride-shares. But more importantly, self-driving Ubers will be a capital intensive model and will have all the flexibility of a taxi, and none of the flexibility of a ride-sharing service.

The long-haul truck driver replacement market is a $100 billion addressable market, about 10 times that of the taxi driver replacement market.

Follow the money.

When Did Expertise Die?

I saw a recent Facebook post of Dr. Tom Nichol's commentary on PBS about The Death of Expertise.

For some unknown reason, Nichols blocked me on Twitter, so I cannot provide this opinion directly. That is his loss.

But Nichols accurately posits the rise of the public Internet has created the side effect of everyone thinking they individually are an expert. However, individuals believing themselves to be experts is only half of the equation. The other half is the discrediting of the true experts, and I believe that happened about a decade or more before the rise of the public Internet. There is a third point, which is the rise of the well known pseudo-expert, and in some cases the celebrity pseudo-expert, such as Jenny McCarthy in the Anti-Vaxxer movement, and Rosie O'Donnell in the 9/11 Truther movement. Celebrity pseudo-experts provide credibility to lay pseudo-experts such as the producers of the original "Loose Change" 9/11 Truther film.

But back to the second point, the discrediting of true experts, or "when expertise died".

In 1989, while in college, I had a roommate who was a journalism major. At that time, they were teaching journalism students expertise is a subject was inherently biasing, and that the opinions of an expert in a subject must be balanced with an opinion of someone who was not an expert in the subject.

He later worked on a story on management, and interviewed an expert in the subject, who happened to be a management professor I worked for as a graduate assistant. He had to then find rebuttal information not from another management professor, but from someone completely unrelated. To me, this was surreal, because I knew both the interviewer and interviewee, and had no reason to question the good intentions of either.

But later it all made sense to me. I grew up watching expert reporters: Jules Bergman, ABC's science reporter; and Irving R. Levine, NBC's economics reporter. I also noticed those expert reporters completely disappeared in the 1980s. Except for the doctors the networks use as medical correspondents and the aviation expert they bring in for airplane crashes, there are no expert reporters any more. I also remember every time in the 1980s we launched a Space Shuttle, the various national news anchors would state the Soviet Union's public statement opinion about the purpose of the mission, as if it was as valid as NASA's stated mission objectives, or as if NASA's stated mission was as invalid as the Soviet's opinion. This latter point goes straight to my original point about my what my roommate was taught: NASA is an expert on their space missions, their opinion must be balanced. Was the Soviet statement credible? Was it valid? Was it simply propaganda? It didn't matter. Was NASA's statement credible? Was it valid? Was it simply propaganda? It didn't matter. To the media, the Soviet position was just as valid as NASA's position. Propagandists at the Kremlin were just as valid as rocket scientists in Houston.

From a purely pop-culture standpoint, I think we tended to believe Jules Bergman on science issues because his name sounded similar to science fiction writer Jules Verne's. I think we believed the bespectacled and bow-tied Irving R. Levine because he fit our visual of what a college economics professor should look like. They were journalists, and not scientists or economists, and they fit a persona, but they were experts in their field as far as journalism went. They had connections, they could get a meeting with the real experts, they had developed a working expertise on their subject, and they had credibility with the public. But they are gone now, and have been for about 40 years.



So I think before we blame the general public, driven by curiosity, and enabled by the Internet (be it WebMD, Wikipedia, or "FakeNews"), we need to consider nature abhors a vacuum, and realize the television media created a vacuum when it cut out those quirky expert reporters, and promoted skepticism and outright distrust of expertise.

Everything I need to know about NetApp’s All-Flash Storage Portfolio I learned from watching College Football

Okay, silly title. I got the idea when Andy Grimes referred to NetApp’s all-flash storage portfolio as a “Triple Option”. To me, when I hear triple option, I think of the famous Wishbone triple option offense popular in college football in the 1970s and 1980s. And that got me to thinking of how NetApp’s flash portfolio had similarities to the old Wishbone offense.

The Wishbone triple-option is basically three running plays in one. The first option is the fullback dive play. This is an up the middle run with no lead blocker. It is up to the fullback to use his strength and power to make yardage. The second option is the quarterback running the ball. While most quarterbacks are not great runners, the real threat of the quarterback in running offenses is the play action pass, where a running play is faked, but the quarterback instead passes the ball. In today’s college football, while the Wishbone may have faded, option football remains popular, and many of the most exciting players are “dual-threat” quarterbacks who can both run well and pass well. But, back to the Wishbone. The third option is the halfback, an agile, quick running back who often depends more on his ability to cut, make moves, and change direction to make the play successful.

In considering this analogy, I wanted to find the right pictures or videos of Wishbone football to make the comparisons to NetApp’s flash portfolio, but found the older pictures and videos from the 1980s to not be that great. So I decided to take the three basic concepts: The powerful fullback, the dual-threat quarterback, and the agile halfback and look at more recent examples. I just happen to use examples from my alma mater, Auburn University, because I knew of a few plays that visually represent the comparisons I am about to make.

So first up is the fullback. The fullback is all about power. It is not about finesse. The fullback position is not glamorous. The fullback had to have the strength to face the defense head-on. To me, the obvious comparison in the NetApp flash portfolio is the EF-Series. The EF is all about performance: Low latency, high bandwidth, without extra bells and whistles which can slow other platforms down.

While I don’t have a good fullback example, I have a similar powerful running back demonstrating the comparison I am trying to make. Here we see Rudi Johnson on a power play break eight tackles and dragging defenders 70 yards to a touchdown from the 2000 Auburn-Wyoming game.

Rudi Johnson great 70 yard TD against Wyoming 2000

The next comparison is to the dual-threat quarterback. The dual-threat quarterback can run or pass with equal effectiveness. In NetApp’s flash portfolio, the obvious comparison is the All-Flash FAS (AFF), the only multi-protocol (SAN and NAS) all-flash storage array from a leading vendor. The multi-protocol capability of AFF (Fibre Channel, iSCSI, and FCoE SAN; NFS and SMB NAS) allows storage consolidation, and truly brings the all-flash data center to reality.

The play which best demonstrates the dual-threat quarterback’s potential is the run-pass option (RPO), where a quarterback rolls out and can either keep the ball and run with it, or pass it to a receiver if the receiver is open. Here we see Nick Marshall on an RPO play which tied the 2013 Iron Bowl with 33 seconds left in the game. The reason the play worked is Nick Marshall, a gifted runner, had already run for 99 yards including a touchdown.

2013 Iron Bowl: Marshall to Coates

That brings us to the halfback, also known as the tailback, or just the running back. For the sake of this discussion, and keeping with the original Wishbone concept, I will use the term halfback. The handful of teams who still run a variation of the Wishbone (Georgia Tech, Navy, Army, Air Force, and a few others), tend to use smaller, more agile athletes as halfbacks. These running backs usually get the ball on the outside, and leverage their agility to make the defenders miss. When I think of agility in flash storage, I think of SolidFire. Agility is a key feature of SolidFire. It scales with agility, provisions with agility, adapts with agility, and is the best storage for agile infrastructures like private clouds, especially private clouds using OpenStack. The best recent example I have seen of a running back leveraging agility to make a play is this run by Kerryon Johnson against Arkansas State.

Watch Kerryon Johnson's incredible touchdown against Arkansas State

So enough fun for now. But if you have a dedicated application needing performance acceleration, such as a performance critical database, NetApp’s EF-Series might be your tackle-breaking fullback powering through spaghetti code and getting the job completed despite the challenge. If you are looking to move to an all-flash data center and need consolidated flash storage to accelerate iSCSI MS-SQL databases and NFS VMware datastores on the same infrastructure, AFF is your dual-threat quarterback. And if you are looking to deploy a private cloud with the agility to grow with your workload, SolidFire is your agile halfback.

On Disruption

A few months ago, there was an email thread at my employer asking the question if All-Flash Storage was a “disruptive” technology. Disruptive, in the business sense, refers to Clayton Christensen’s definition of the term from his book, “The Innovators’ Dilemma”.

This, from a year ago, Christensen reviews his concept:

What Is Disruptive Innovation?

However, I think this is a narrow, and perhaps obsolete definition. He says Uber is not disruptive, because it did not originate in the low-end or new-market segments. However, while Uber did not disrupt car for hire, it did disrupt the capital model of cars for hire, and it did disrupt the medallion licensing model. Then the article also talks about how Netflix, in its original format (DVDs by mail) attacked an underserved periphery—not the low end, and not a new segment—of the market.

If we use the pure Christensen definition, All-Flash Arrays (AFAs) are not disruptive, but HyperConverged Infrastructure meets the definition. But perhaps we should look more broadly at the definition.

“The Innovator’s Dilemma” is 20 years old. It was written during the Dot-Com boom. Business books are not canonical. If they were there would never be revisions and follow-ons.

I think we need to take a wider view of disruptive technologies. Uber disrupted car for hire capital and licensing models. Driving an Uber is much less expensive than buying a taxi medallion, so the cost of entry was disrupted.

So how does that apply to AFAs? We know cost of IOPS is much lower with AFAs. We also know the costs of sizing and performance management dramatically decrease. One can argue the TCO of AFAs is lower. While AFAs did not enter at the low-end or a new-market segment, it did enter at a periphery, at a market segment (high transactional performance storage) where it offered a lower cost. AFAs disrupted a market segment of the overall frame storage market. Not the Mainframe attach segment, and not the extreme reliability segment, but at the assured high performance segment.

But here is another aspect of AFAs I am seeing—they mandate changes to a customer’s operational model. AFAs were made cost effective in part by using data reduction technologies (deduplication and compression). While there were some hard-drive based storage arrays which leveraged data reduction technologies (NetApp FAS, EMC Celerra, Sun/Oracle ZFS based arrays), these data reduction technologies were not available on high-end frame storage (EMC Symmetrix/DMX/VMAX, HDS USP/VSP, IBM ESS/DS8000). These data reduction technologies worked well for certain workloads: virtual machines benefited from deduplication, and OLTP databases benefited from compression.

This meant AFAs with built-in data reduction, targeting small, peripheral workloads (VDI, high-transaction OLTP), were set up for easy success.

However, at the same time other trends were occurring. To more effectively leverage the expensive high-end frame storage, some DBAs were turning on compression within their database software. Yes, this increased the number of CPUs needed to run the database, and increased their cost, but often DB licensing was a sunk cost. It was also possible to compress at the OS/filesystem level. It was not unusual in organizations where IT departments charged back storage capacity to users, for users to turn on compression in their servers to reduce their chargeback.

The second thing that happened over the last five years has been the fear of a data breach. This has driven the need to encrypt data at rest. While storage arrays offer this capability through Self-Encrypting Drives, encryption boards, or software encryption running on the array’s controller, often enabling storage encryption could only be done after upgrading the storage array to a new model. As a result, turning on encryption at the application level (i.e., the database), at the OS level (encrypting file systems), or at the VM level (using products like HyTrust) was a much faster path to security for many customers. Also, customers were assured only host level encryption ensured data was encrypted “over the wire” in addition to at rest.

The result of either of these technologies is it eliminates ability of the data reduction technology in the storage array to provide any benefit, and it returns the cost per gigabyte of flash storage to what it was with early generation, non-efficient architectures, which ultimately lost out to the AFAs with built-in data reduction.

The only way to benefit from an AFA’s data reduction features are to ensure applications and operating systems are not running host level compression or encryption. It may mean ripping out products like HyTrust and Vormetric. It may mean internal battles with DBAs. It may mean new terms and conditions in internal SLAs and storage chargebacks. The All-Flash Data Center sounds innovative on paper, but implementing it means working across traditional IT divides of applications, servers, security, and storage.

There are some data types which are natively compressed. For example, all the current Microsoft Office file formats are compressed. Additionally, most image files are compressed. Traditional file shares full of PowerPoint files are not going to benefit from AFA data reduction. Generally these workloads have never rated high-performance storage, and because of the lack of reducible data, it will take more time for the cost per gigabyte of All-Flash storage to come down to a point to provide the necessary payback to justify migrating these workloads to flash.

Why did I go down this path? It was to point the potential limits of a disruptive technology. When AFAs were narrowly applied to certain workloads, there was a cost-benefit which accelerated their adoption. When they are applied more broadly, they hit organizational barriers to adoption. Perhaps these barriers mean AFAs do not fit the definition of a disruptive technology. However, in IT I see many “disruptive technologies” which ultimately force significant operational changes on IT organizations. That was true for UNIX, Storage Area Networks, Windows, Linux, and VMware. It will likely be true for All-Flash Storage, Software Defined Networking, and adoption of Cloud Computing.

Why is There Not More Scepticism on Climate Science?

I continue to be surprised at how many people, especially Millennials (who are supposed to be skeptical), take "Climate Change" as gospel, despite evidence of highly questionable, and in some cases fraudulent science, such as the math used in Mann's "Hockey Stick" formula, and other questionable science revealed in the East Anglia email leaks.

Here are the questions I pose to anyone on the topic:

• What percentage of warming is due to CO2 emissions due to the burning of fossil fuels?
• What percentage of warming is due to other man-caused reasons?
• What percentage of warming is due to changes in solar activity?
• What percentage of warming is due to changes in other natural reasons?

Given observed questionable surface temperature measurement stations, and a noticeable difference in surface station temperatures and atmospheric temperatures, do Climate Scientist's heavy dependence on surface temperature measurements lead to unreliable results?

Source: New study shows half of the global warming in the USA is artificial

Source: 7 questions with John Christy and Roy Spencer: Climate change skeptics for 25 years

Given many Climate Scientists claim solar activity plays no significant role in Climate Change, but other Climate Scientists claim the significant pause in global warming is due to a decline in solar activity, how trustworthy is the climate science regarding solar activity?

Source: New study claims low solar activity caused "the pause" in global temperature – but AGW will return!

Source: Tiny Solar Activity Changes Affect Earth's Climate

Given one can insert random numbers into Michael Mann's equation and still produce a "Hockey Stick" output, how trustworthy should Dr Mann's science be considered?

Source: Michael Crichton - On Michael Mann's Climate Temperature Graph

Given evidence scientist Keith Briffa selectively picked evidence to support his desired outcome, and discarded evidence which did not support his desired outcome, how trustworthy should Dr Briffa's science be considered?

Source: YAD06 – the Most Influential Tree in the World


Given evidence scientist Philip Jones stated he used Michael Mann's "trick" to "hide the decline" of late 20th century cooling to overstate warming in the industrial era, how trustworthy should Dr Jones's science be considered?

Source: Climategate reveals 'the most influential tree in the world'

Source: IPCC and the "Trick"

Given climate scientists refused to allow critical peer review of their research, and only allowed it to be peer-reviewed within their tight circle of fellow climate scientists who believed the same way they did, how trustworthy should their science be considered?

Source: The tribalistic corruption of peer review – the Chris de Freitas incident

Given climate scientists working at government organizations refused FOIA requests for details of their research, how trustworthy should their science be considered?

Source: Climategate: James Hansen Finds Complying with FOIA To Be Too Much of a Burden


So there it is. Why not more skepticism, not that temperatures are rising, but skepticism of the science? I have said repeatedly, Climate Science is a Social Science, not a Physical Science. It is more about computer methods and curated data, and less about measurement. And other Social Science is held to much greater skepticism than Climate Science.

UPDATE:

Now there is this. The data used to dispute the "pause" in Global Warming is in dispute. By definition, science based on data that is in dispute cannot be considered "settled".

Exposed: How world leaders were duped into investing billions over manipulated global warming data

"True" Private Clouds

Wikibon is talking about "True" Private Clouds. I think their definition is too narrow, and gets into the weeds. It misses the true customer of a "true" private cloud. And there are two customers. The first is the organizational customer that purchases a private cloud. The second is the internal end-consumer of cloud services.

To Wikibon's credit, the definition of "Private Cloud" is an issue that needs to be addressed. In my career I have seen too many organizations overuse the term "Private Cloud". I have seen a VMware cluster deployed on disparate hardware with no upper level cloud management platform called a private cloud. I have seen converged infrastructure, acquired but managed identically to non-converged infrastructure (as discrete components each managed by their functional staff) called private clouds.

Converged infrastructure plays a role in a private cloud, be even that term is challenged. I have seen disparate servers and storage, purchased separately at different times, cobbled together and called converged infrastructure after the fact. I have also seen single-SKU converged infrastructure broken apart, support for component infrastructure separated, and individual components upgraded on different life-cycles.

From an operations perspective, I have seen mature IT organizations in large enterprises provide similar levels of managed services as traditional managed service providers. I have also seen the converged infrastructure single-support model dramatically fail organizational customers, and provide no better single support that that provided by an reseller or managed service provider.

If the goal of a "true" private cloud is to provide a similar level of service offering to internal end-consumers they receive from a public cloud, but with higher levels of compliance and data sovereignty, then much of the detailed requirements Wikibon mentions are not necessary. As long as the organization can provide an offering to internal end-consumers which is competitive (on cost,  ease of consumption, and reliability), it should meet the definition.

Here are what I believe are required of a "True" Private Cloud:

• Acquired in consolidated units of management, virtualization, compute, network, and storage with common amortization, and common life-cycle management.
• Components supported as an integrated whole, with a single number, first-call support model, and escalated support abstracted from the internal end-consumer.
• Compute, storage, network, and virtualization managed as a single entity by a single, cross-functional team.
• Provisioned and managed via a cloud management platform (CMP).
• Consumed by internal end-consumer as a shared resource in logical, not physical increments, i.e., VMs and GBs.
• End-consumer offerings include multiple performance and data protection SLAs.
• Provides charge-back to internal end-consumers.
• Provides the Private Cloud operator performance, capacity, and licensing budgeting of the infrastructure; performance metering and capacity measurement to manage over-subcription, prevent over-consumption (especially of performance), and allow for elastic performance and capacity scaling; and provide built-in performance and capacity planning for predictable infrastructure growth.
• Managed by high IT maturity organizational customer IT staff, or optionally part of a managed services offering  that does not require organizational customer IT staff to manage.
• Financed to organizational customer either through capital purchase, capital lease, operational lease, capacity lease, or pay-per-use offering.

Some organizational customers will want to capitalize the "True" Private Cloud and manage it themselves. Others will want to basically rent the whole stack to include the software, and have it managed for them. But the common denominator should be how the internal end-consumer consumes the offering. It should look, feel, and cost as much like the public cloud as possible.

U.S. Cyber Command's Requirements Demand Warrant Officers

Yesterday there was a hearing in front of the House Armed Services Committee, "Outside Perspectives on the Department of Defense Cyber Strategy".

Some of the points brought up were about the personnel management of military cyber warriors. This is a challenge, because "cyber" (BTW, I HATE the term) is both an infrastructure (i.e, IT Infrastructure), and a domain (Information Warfare). It is an area where both warriors and janitors walk, more akin to urban warfare than other historic domains.

Since the late 1980s, the military has treated IT as an area where COTS technologies should rule, to both increase the productivity of the military, and to reduce operating costs. At the same time, the PC and client-server boom of the 1990s drew skilled IT technicians from the military to the higher paying civilian sector.

Through the 1980s, the military, had its own data, uniformed, processing specialists. The father of a high-school friend was a Technical Sergeant in the Air Force and a Burroughs mainframe programmer.  In the 1990s, most of the programming positions were either converted civil service, or outsourced to contractors.

The second wave occurred in the 1990s with the decentralization of IT acquisition, management, and support from central service commands (i.e., Air Force Communications Command) to the local military bases.  This was followed with A-76 studies converting many base level IT jobs to a combination of civil service management and contractor work forces.

The result of all of this is the military lost its uniformed expertise in information technology.

Fast forward to today, and information infrastructure is as much a domain in warfighting as the seas and the air, yet the military is left without the skills in uniform which correlate not only to captains of ships and pilots of airplanes, but also to the technicians, operators, and maintainers. As a result, the military has once again centralized IT acquisition, management, and support, and is once again filling positions with uniformed personnel.

However, IT skills is unique in several ways. They are perishable. Old skill requirements (i.e., Novell NetWare, UNIX) become obsolete and unneeded, and are replaced with new skill requirements (i.e., Windows Server, Linux). To ensure quality, they require validation (i.e., IT Certifcations). Because they are COTS based, they are inexpensive compared to unique military skills. They are fungible and readily transferable to the civilian sector.

Another unique aspect of the military is ab initio training. The military will take someone from high school with the appropriate aptitude, enlist them, and train them up to a level of reasonable, beginners level, productivity. It then will use on the job training and continuing education to build expertise. In the case of an in demand skill set, this creates issues with retention. This is a bigger deal than a military turbine engine mechanic--there are only a handful of airlines needing them. But almost every organization needs a Windows administrator.

Then there are the challenges. The military needs, smart, highly skilled, problem solvers for day to day operation of the IT infrastructure. The military information infrastructure is more likely to be attacked both in peace and in wartime, but rapid recovery is critical in wartime. Poor retention hurts this need. The military needs deeply skilled, highly experienced IT technicians. But the need for operational managers is not that great, so the college educated, commissioned officer corps is not the appropriate career path for an IT technician. Something else is needed.

The military position of Warrant Officer is that of an technical specialist. Historically, the technical expertise came from experience serving in the enlisted corps.  In modern times, the Army uses Warrant Officers as helicopter pilots and trains them to the appropriate level or technical expertise.

Warrant Officers can serve as highly skilled individual contributors or as first level managers. It would seem a perfect career path for an enlisted military IT specialist. Tie it to certifications, and perhaps an Associate Degree, along with a service commitment and a retention bonus.

On the commissioned officer side, the career plan should be more on IT architecture, Information Warfare and advanced academic education. College educated officers would start focusing on both supervisory roles, and architectural roles. Then the focus should be on an advanced degree in the appropriate field of study. From there, moving to an Information Warfighting planning role, followed by the appropriate mid-career professional military education. Cross flow between related fields such as military intelligence would also be appropriate, however, this should be treated with care, as military intelligence often recruits from liberal arts studies such as history, foreign language, and political science. A cross flow program should not disrupt either the military intelligence corps, or the information warfare corps. Finally, the Joint Forces Staff College should create a dedicated Command and Staff school for information warfighters, with the goal of creating cadre of information warfighting leaders for all of the services.

Ultimately, the combination of a cadre of commissioned information warfighting leaders, combined with a corps of highly skilled warrant officer information warfighting specialists, would go a long way towards developing the cyber warrior force our nation requires.

A Reply to Chris M. Evans' "The NetApp Conundrum"

Storage blogger Chris M. Evans wrote a recent post on LinkedIn entitled "The NetApp Conundrum".

https://www.linkedin.com/pulse/netapp-conundrum-chris-m-evans

As a NetApp employee, and long-time member of the IT vendor industry, I have provided the following response.

I am having trouble resolving two points you made Chris. One is Data ONTAP is old (23 years, to be exact), and storage architectures only last 20 odd years. The second is clustered Data ONTAP is not Data ONTAP (the 23 year old one), but a new and different product created by merging some of Spinnaker's technology (acquired in 2003), with some of NetApp's technology in 2009. By my math, that makes clustered Data ONTAP six years old, and by your own calculation, it has 14 years of longevity left.

A few other points:

It is impressive HDS VSP's SVOS can run on a laptop. I can run a four-node clustered Data ONTAP cluster on my laptop.

The debate over the HA-Pair construct vs. a multi-node HA construct is an engineering and design debate, based on customer requirements, performance, time to market, predictable failure characteristics, and trade-offs--not ideology or perceived elegance. I would note VMAX engines are failover pairs for the same reason we use failover pairs in clustered Data ONTAP. It is also worth noting EMC changed the cache mirroring approach in Isilon with its Endurant Cache to a logical cache pair construct to maximize performance. Similar to clustered Data ONTAP, EMC XtremIO uses a cluster constructed of failover pairs, and Pure Storage use a failover pair scale up architecture similar to NetApp 7-Mode or EMC VNX.

True scale-out, distributed storage is interesting, but it presents challenges in developing fast, reliable, predictable failover. It is also very difficult to implement highly efficient data protection schemes, such as parity, double/triple-parity, and erasure coding in such an architecture. There is a reason Hadoop clusters, VSAN clusters, and Nutanix clusters use mirroring and triple mirroring for data protection. Nutanix's just announced EC is only for cold data.

What is happening today is almost all of the new all-flash array start-ups (XtremIO, Pure, Kaminario, Whiptail/Cisco, and Nimbus Data), and hybrid array start-ups (Nimble, Tintri, and Tegile), use log-structured filesystems, non-volatile memory and write coalescing, write to free-space, and parity RAID algorithms as the basic underlying technologies for their arrays. These concepts are more than 20 years old. NetApp built WAFL and Data ONTAP on these concepts more than 20 years ago because they worked. And they still work today, especially for NAND flash media. That is why NetApp continues to improve and develop Data ONTAP. Because the alternative to Data ONTAP looks a awful lot like Data ONTAP. Don't take my word for it--just look at the recent hybrid and all-flash storage players out there.

Thoughts on the HP Split

Too many people equate the PC business side of current HPQ as Compaq, and the enterprise side of current HPQ as the old HP. The truth is the old HP was nearly dead as a enterprise computing products company after spinning out Agilent and before acquiring Compaq. A quick look at HP's current technology portfolio shows much of it came in through acquisition. Much original HP technology has faded away. What is worse is much of HP's acquired technologies have been neglected to atrophy.

All of HP's current x86 server technology is former Compaq technology. The HP c-Class Blade System is a Compaq design which was in the works prior to the acquisition. HP's rack-mount x86 server technology is former Compaq. Engineering for HP x86 servers is done at the former Compaq facility in Houston.

Prior to the Compaq acquisition, HP's x86 server business was struggling to compete with IBM and Compaq's x86 server offerings. HP's x86 servers suffered from product quality issues, and little innovation.

HP's enterprise storage portfolio was a joke prior to the Compaq acquisition. Their organic mid-range system was sub-par, and they relied on an OEM relationship with EMC for their high-end solution.

Through the Compaq acquisition HP acquired the most sophisticated mid-range SAN platform of its time, the Enterprise Virtual Array (EVA). This was developed by Digital's StorageWorks division, which was working on the EVA prior to Compaq's acquiring them.

Within a decade, HP failed to innovate the EVA, and had to acquire 3PAR (and pay three times its market price due to a bidding war with Dell), to reinvigorate its mid-range storage line. HP also acquired LeftHand Network's SMB iSCSI systems to address the low end of its portfolio. HP still relies on an OEM relationship for the high-end, but now with HDS.

HP divested itself of the microprocessor business, ceding its HP-WideWord VLIW design to Intel to become the Itanium EPIC processor.

In the enterprise server space, HP's acquisition of Convex Computer gave it the SuperDome system, which originated as Convex's next generation Exemplar. While HP has iterated and evolved Convex's NUMA interconnect several times, there has been no net-new high-end server design from HP. The SuperDome 2 simply marries the "Convex Exemplar++" interconnect with the Compaq c-Class I/O backplane. And the idea that the coming "x86 SuperDome" will be anything other than a niche system is not going to happen.

In operating systems, other than its "Ignigte" bare-metal provisioning technology, HP-UX has lagged technologically behind Solaris and AIX for two decades now. HP's "innovations" were to OEM Veritas filesystem and volume management technology.

In automation, HP acquired OpsWare, the best technology out there in 2007. But now all of the oxygen in data center automation is being sucked up by either VMware or OpenStack.

HP had an excellent managed services organization (it used to be headquartered here in Atlanta), but this organization was subsumed into whatever is left of the former EDS post acquisition.

So the only organic components of HP I see still having value are the 30% of HP Services which was not part of the EDS acquisition and HP's printing division. Hewlett-Packard Enterprise is little more than a publicly traded private equity fund, a holding company of various technology brands (Tandem/DEC/Compaq/3PAR/OpsWare/EDS), in the mold of CA Technologies. In that way, they are similar to IBM, which also has acquired and failed to maintain many technologies. The difference is, IBM's organic enterprise technology (Mainframe, POWER, DB2, etc.) is aggressively maintained.

I honestly think the HP PC/Printer spin out will never happen as envisioned. Instead, HP will likely sell of HP PC/Printer to a private equity company who seeks the printer division as a cash flow business, and sees the PC division as something they have to buy in order to get the printer business. They will likely sell the PC business to an ODM who seeks a branded entry.

Where was I?

Where was I? It seems everyone is answering this question.

I worked for Sun Microsytems at the time and was in the King and Queen building complex in Atlanta in a sales training class. There were no TVs, so we only go the news via cell phones and the Internet. But the Internet had ground to a halt.. As soon as it happened, I knew it was Bin Laden. I was convinced Bin Laden (not Iranian Hezbollah) was behind the Khobar Towers bombing, which killed five of my 71st Rescue Squadron mates in 1996. I felt it odd to be explaining Bin Laden (who I described that day as the closest thing to a James Bond super villain who actually existed on this earth), continuance of government, and SCATANA to my coworkers. It was like I was in on everything which was happening and everyone else was blind. Somewhere in there I called my Reserve unit in Alabama and let them know if they needed me I could be ready and down there in four hours.

At some point someone said Sun's New York sales office was in the World Trade Center (floors 25 and 26 of the South Tower, the second tower hit, and the first to collapse). That realization changed the dynamic of our class. Within about an hour we got word the entire Sun office evacuated after the North Tower was hit, and everyone in the office made it out safely. Crazily enough, we pressed on with our class. We wandered like zombies to HoneyBaked Ham for lunch, came back, and I presented my portion of the training class.

That evening, after a couple of Jack on the Rocks at Joey D's Oak Room with my colleagues, I drove home. On the drive, I called a former 71ster (Darryle Grimes) stationed at the Pentagon. He had been in the Pentagon during the attack, but was far enough away to not actually feel the impact. He told me the Pentagon had gone to 24 hours operations and he had to be back there in a about an hour.

The second thing I remember is I was not able to sleep that night. That is only one of two nights in my adult life I was not able to sleep at all.

Politicizing Everything

The July 12, 2010 letter from five of the members of the Columbia Accident Investigation Board to Senator Barbara Mikulski is a piece of political, not scientific work, and can only be seen as an attempt to offer a fig leaf to an otherwise naked policy. I do not believe these five people just spontaneously decided to write this letter without being solicited to do so. First, it is a letter from only five of the thirteen CAIB members. Second, those members are claiming to speak on behalf of the CAIB:

"We would be glad to answer any questions that you or other members of Congress may have concerning the CAIB report and its application to today’s space policy issues."

Third, one of the five authors, Shelia Widnall was a Democrat political appointee, and three others, Steven Wallace, Douglas Osheroff, and John Logsdon were all Obama campaign contributors. Without knowing the opinions of the other eight members of the CAIB, these are just the opinions of individuals, and more accurately, potentially biased individuals. Fourth, the letter misrepresents some of the conclusions of the CAIB, specifically the following:

"The design of the system should give overriding priority to crew safety, rather than trade safety against other performance criteria, such as low cost and reusability, or against advanced space operation capabilities other than crew transfer."

"This conclusion implies that whatever design NASA chooses should become the primary means for taking people to and from the International Space Station, not just a complement to the Space Shuttle. And it follows from the same conclusion that there is urgency in choosing that design, after serious review of a "concept of operations" for human space flight, and bringing it into operation as soon as possible. This is likely to require a significant commitment of resources over the next several years. The nation must not shy from making that commitment."

Abandoning Ares I and Orion is being done for cost reasons, not for safety reasons. The primary means of taking people to the ISS will be the Russian Soyuz. Abandoning Ares I and Orion abandons urgency and does not bring a system into operation as soon as possible. It specifically abandons the significant commitment of resources over the next several years. It is shying away from the needed commitment.

Furthermore, the letter misrepresents the Ares I when it compares it to current EELV boosters. The first stage of the Ares I is based on the man-rated Space Shuttle SRB, of which 262 have flown successfully, a fact which escapes the "CAIB Five", because it makes the 34 EELV launches pale in comparison. The J-2X Ares I upper stage engine is based on the man-rated J-2 engine which had a 96% success rate, and despite a handful of engine failures, it had a 100% mission success record.

The Orion spacecraft is a simply scale up of the Apollo Command Module spacecraft. Scaling up an existing design is a proven cost and risk mitigation strategy, and was the same strategy used to develop the highly successful Gemini spacecraft. The Gemini capsule was based on an enlarged Mercury capsule, which allowed engineers to focus on the advanced features of Gemini rather than the capsule itself. This is no different from Orion. Much of the original aerodynamic work done on the Apollo Command Module still applies, so it means a safer, quicker, less costly solution.

Additionally the CAIB noted:

"It is the view of the Board that the previous attempts to develop a replacement vehicle for the aging Shuttle represent a failure of national leadership. The cause of the failure was continuing to expect major technological advances in that vehicle."

Ares I / Orion, by leveraging existing boosters, engines, and spacecraft designs, avoids the expectation of technological advances. Even the decision to move to a splashdown water landing was done to reduce risk and cost.

"With the amount of risk inherent in the Space Shuttle, the first step should be to reach an agreement that the overriding mission of the replacement system is to move humans safely and reliably into and out of Earth orbit. To demand more would be to fall into the same trap as all previous, unsuccessful, efforts."

While the Constellation project encompassed more than simply transporting astronauts to orbit, the Ares I / Orion system was focused only on this. The only additional demand was that a future uprated version of Orion, carrying four astronauts rather than six astronauts, be capable of flying to lunar orbit, be parked unmanned in orbit, and later return to Earth. Most of these capabilities would impact Orion's service module, not the manned capsule.

"Continued U.S. leadership in space is an important national objective. That leadership depends on a willingness to pay the costs of achieving it."

It is clear President Obama does not have the will desired by the CAIB, and Obama's decision represents another failure of national leadership. It also seems the "CAIB Five" no longer agree with the importance of U.S. leadership in space. This letter can only be seen as a dissent from Chapter 9 of the original CAIB report. The authors should be vigorously challenged not only on their statements in this letter, but also on their support of the original CAIB report's conclusions.

x86 Rises, Part 4: The emergence of Linux as a viable datacenter OS

Several years ago I drafted a white paper I called "x86 Everywhere". I started it in the fall of 2004, let it sit, and updated it in April 2005. It remains unfinished, but with the release today of Intel's Nehalem processor, I took a look at it again. Here it is:

Three trends could allow what I call "x86 Everywhere" to happen.

The third trend necessary for "x86 Everywhere" is the possibility of the emergence of Linux as a viable datacenter OS.

This seems less likely than high-end x86 servers at this point, but it is certainly possible in several years time, if the efforts of the Datacenter Linux project bear fruit. Windows on 32-bit x86 systems did not penetrate the datacenter, in part because the hardware was not 64-bit, the hardware was not scalable, and customers did not trust Windows with their critical data.

Today, the hardware is 64-bit, AMD Opteron is scalable to eight-sockets today, Intel is pursuing efforts that will likely address the scalability limitations of Xeon, both AMD and Intel are aggressively pursuing multicore chip strategies, and customers trust Linux in places they formerly only trusted UNIX. The result is a very real, industry standard ABI/ISA platform combination that scales from embedded systems, to an inexpensive developer platform (the PC), to midrange enterprise datacenter computers. This could be enough to cause a tipping point, creating a fundamental driver for the Datacenter Linux initiative. Such a change in the primary enterprise compute platform from RISC/UNIX to x86/Linux would likely be highly disruptive to the industry, and would rival the move of commercial computing in the early 1990's from proprietary minicomputers to SMP RISC/UNIX servers. Once established in the datacenter as a viable midrange enterprise platform, like SPARC/Solaris it becomes a straightforward scaling exercise for x86/Linux to establish itself as a high-end platform.

Finally, while not a trend driving large scale x86 adoption, there are other developments to consider. Intel has a virtualization technology, called Vanderpool on desktops and Silvervale on servers, that will help provide partitioning on its systems. AMD has also stated it intends to offer a virtualization layer, called Pacifica. AMD has also stated it plans to improve RAS features of its Opteron, and it is likely Intel will do the same with Xeon, using features it already offers on Itanium. Both of these key technology areas will improve adoption of x86 servers in the enterprise market.

How will this play out?

First, Dell's strategy is to only enter established markets, and to do so with a superior fulfillment system. For markets that are not at that point, Dell has used partnerships, such as its existing partnership with EMC. Dell also partners with Unisys to resell Unisys' 8-way Intel Xeon systems. Therefore the most likely path for Dell is to primarily continue the status quo, assuming four socket x86 systems and below represent the lion's share of the server market. If there is a need to address the greater than eight-socket x86 server market, Dell could expand the Unisys agreement beyond 8-way. If Dell expands into the Opteron market, and needs to address the greater than eight-socket x86 server market, it could partner with Newisys (also an Austin TX company).

IBM already is a player with its Enterprise X Architecture (EXA) for Intel systems. However, IBM has close ties to Newisys (the founder is ex-IBM, and the Horus chipset is based on similar principals to EXA), IBM sold its North Carolina based PC Server manufacturing plants to SCI-Samna, IBM has a strong presence in Austin TX, Newisys' home, and IBM has strategic agreements with AMD around CPU fabrication technology. It is possible IBM could offer the Newisys system in addition to its own EXA systems.

HP is committed to x86 in the four-socket and below space, and is a strong backer of Linux. If the x86/Linux platform gains momentum, it would simultaneously weaken Itanium sales. This would require a strategy change for HP, but such a change would be necessary to remain a viable datacenter systems vendor. To address this, HP could OEM a solution if needed to address a short term requirement. HP did this with NEC's high-end Itanium system before HP adapted its Superdome system to accept Itanium processors. Here the most likely partner would be Newisys, with similar Texas roots to the Compaq, whose former Texas offices server as headquarters for HP's x86 division in the post-merger HP. Longer term, HP's relationship with Intel could produce a high-end x86 system, especially given the common chipset Intel promises for Itanium and Xeon. In fact, HP's “Arches” system, the follow-on to Superdome, could easily accept future Xeon processors, given the common Itanium chipset. HP could also acquire a solution, but the most likely acquisition in this case would be Unisys. A Unisys acquisition would be defensive as well if Unisys had or was considering a significant Dell agreement.

Sun has some of the closest ties to AMD, and Sun has the technology to build large systems. Sun already plans eight-socket Opteron systems. If a significant market for larger than eight socket x86 servers emerges, Sun will have to decide how to address that market. However, balancing the high-end SPARC and x86 business would be a challenge for Sun. If the scalable x86 market shows great promise, the best technical solution for Sun could be an even tighter AMD partnership with technology sharing to allow common systems to be built with either AMD or SPARC processors. The potential for Sun to leverage common technologies such as coherent Hypertransport for SPARC systems as well as Opteron could offer considerable economies of scale. This could make the most sense in the post APL timeframe. A secondary solution, which also offers a near term solution, would be an OEM deal with Newisys. Sun has relationships with SCI-Samna, OEMing Newisys' two socket and four socket Opteron servers as the V20z and V40z, and Sun contracts with SCI-Samna to manufacture low-end UltraSPARC servers. A deal with Newisys around higher-end systems would also server to more strongly establish Sun in the Texas information technology community, clearly one of the top IT centers in the world, and the most important in the x86 business.

AMD's best interests are served if it does not depend on other vendor's chipsets for scalability. Therefore, offering a higher-end Opteron processor with more coherent Hypertransport links allowing greater glueless SMP scalability is the most likely path for AMD.

Similarly, Intel's best interests are served if it can offer everything needed to build a scalable server directly to the distributor. This is the shift needed to move high-end servers into the commodity space, and allow Dell to enter the market with superior logistics.

Based on all of this, a two-phased industry approach is likely. The first being server-vendor based proprietary scalable solutions (such as IBM's EXA, Unisys' CMP, and Newisys' Horus), followed by processor vendor solutions based on in-chip features.

Who is threatened most by x86 Everywhere? One could say Sun, who relies on SPARC systems for the vast majority of its revenues. However if x86 Everywhere happens, SPARC's installed base is still very large, and will not be replaced overnight. The bigger victim is likely IBM, who is trying to repeat Sun's SPARC success with its POWER architecture. In fact, assuming a Sun/AMD partnership could allow Sun to build SPARC or Opteron systems from common technology (i.e., memory controllers and memory subsystems, coherent Hypertransport MP interconnects, and common Hypertransport I/O bridges), SPARC systems could be continued as long as customer demand supported the design of SPARC processors.

The big loser in this appears to be Newisys. SCI-Samna's business model is two-fold: Contract manufacturing and OEM manufacturing. Newisys' low-end systems fit well in the OEM model, and SCI-Samna has had success selling these systems to its OEM partners. However, the high-end Horus systems do not fit the OEM model. Several have tried OEMing datacenter servers, and few have succeeded. In the late 1990s, Unisys OEMed its x86 CMP system to both Dell and Compaq. The Dell OEM lasted only months. Dell realized a 32-way datacenter server did not fit its direct business model. Compaq's deal lasted a little longer, but it too abandoned the OEM arrangement. Other OEM deals include HP's OEMing of NEC's first generation Itanium system, which delivered few sales. The most successful OEM deal of datacenter servers appears to be Bull Worldwide's OEMing of IBM's pSeries servers, but this arrangement created significant channel conflict for IBM in europe, and seems to always be in danger whenever IBM announced a new generation of RISC/UNIX servers. Fujitsu's deal with Siemens is not considered as an OEM deal here because it is really more of a partnership. The Fujitsu-Siemens model is worth considering by Newisys, as it is a successful model of a business relationship between a high-end server manufacturer and a IT solutions provider. The most likely target customers for Newisys' Horus system are IT integrators such as EDS. IBM has a high-end x86 server in its product portfolio. EDS does not. IT integrators can provide the professional services required in selling such systems. Also, because this would be an OEM arrangement, there is the opportunity for greater margins and services to the IT integrator, compared to deals which involve simply reselling an server vendor's product.

x86 Rises, Part 3: x86 Grows in Performance and Scalability

x86 Rises, Part 2: Decreasing Value of Big UNIX

x86 Rises, Part 1: The Background