In Silicon Valley, one firm stands out above the rest for the way it helps organizations use data for the betterment of their operations: Palantir.
Based in Palo Alto, the firm was founded in 2004 by Silicon Valley investor and PayPal founder Peter Thiel. Its customer contracts totaled $1.1 billion in 2014 - up from about $30 million in 2009, meaning its annual growth rate is 107%. It now has roughly 2,000 employees and is worth an estimated $20 billion, making it the third richest venture-backed company in the United States, behind only lift-sharing app Uber and accommodation service Airbnb.
Despite this, relatively little is known about the company. There’s a certain irony in just how privately Palantir operates given that its central reason for being is to find things out about people. However, leaked documents and a raised public profile mean that we now know far more than we did a few years ago.
Palantir is named after the seeing stones in ‘The Lord of the Rings’ that granted powerful people the ability to see the truth from afar. Its toolsets are used to analyze massive data caches, and they primarily target the three industries that have the largest datasets to analyze: government, the finance sector and legal research. It is fundamentally an interface that sits on top of existing data sets and displays data to users for analysis, helping litigators and law enforcement to find connections that would otherwise be impossible to spot. Users do not have to use SQL queries or employ engineers to write strings in order to search petabytes of data. Instead, natural language is used to query data and results are returned in real-time.
A Palantir deal can run between $5 million and $100 million, with 20% being asked for up front and the rest being paid only if the customer is satisfied at the end of the project. The company is known for two software projects in particular. Palantir Gotham is the tool used by counter-terrorism analysts, fraud investigators at the Recovery Accountability and Transparency Board, and cyber analysts at Information Warfare Monitor (responsible for the GhostNet and the Shadow Network investigation). Palantir Metropolis is used by hedge funds, banks, and financial services firms.
As of 2013, Palantir was used by at least 12 groups within the US Government. These include the CIA, DHS, NSA, FBI, the CDC, the Marine Corps, the Air Force, Special Operations Command, West Point, the Joint IED-defeat organization and Allies, the Recovery Accountability and Transparency Board and the National Center for Missing and Exploited Children. The CIA’s venture capital arm actually invested $2 million in the project at its inception, and they have been well rewarded, with Palantir software successfully illuminating terror networks. One example of their successes is the cracking of a spy operation dubbed Shadow Network that had, among other things, hacked the Dalai Lama’s e-mail account. It has even been used to track patterns in roadside bomb deployment, and was able to conclude that insurgents were using garage-door openers as remote detonators.
Palantir’s operations are not limited to matters of national security. On the fluffier side, it is also helping Hershey’s Chocolate to learn more about things like where best to put their chocolates in stores, mining data customer transaction and store data to discover out that it sold best when placed next to marshmallows.
Despite its relative anonymity compared with other tech giants, it has not completely managed to avoid controversy, and was forced to apologize when one of its employees floated the idea of discrediting Wikileaks. Despite its involvement with security agencies though, it played no role in NSA’s bugging of citizens, and is a strong advocate for privacy. Its software incorporates a series of safeguards which limit who can see particular data, and it lays ‘audit trails’ for investigators to follow to ensure that the rules were followed.
Palantir has a strong idealogical bent. Wages are capped at $137,000 - a meagre sum by Silicon Valley standards - and the work they do for law enforcement is driven more by a desire to use big data to protect the world from evil. As the company grows and the clamor for it to go public increases, it will be interesting to see how these ideals stand up.