data

A quadrillion lives are in your hands. We often hear people talk of “protecting future generations”,and there is certainly a lot of value in thinking of your children and grandchildren when thinking about the future – it makes it personal.

But there is an additional way of thinking about this, which carries equal moral authority – that of existential risk at a human-extinction level.

If you use a pan-generational lens, the lives of all of the potential future generations are at stake.

Think about that idea for a moment. The future of human history.

It changes our perspective: we are familiar in considering genocide as abhorrent, but we are not used to thinking of omnicide (or ecocide) as even a credible threat.

You can also contextualise this by thinking of the differences between (a) peace, (b) 99% extinction, and (c) 100% extinction. What are the relative differences between (a), (b), and (c)?

Arguably the Manhattan Project was the first time we’d formally assessed the potential for omnicide – the study looked at whether a nuclear blast would create a chain reaction in our atmosphere, potentially destroying all life.

With an economic lens, we could consider our current financial markets as a “flawed realization” – we may have reached a technological maturity, but our financial infrastructure may be dismally and irremediably flawed: and a systems change needed to remedy it. It certainly has succeeded in ephemeral realization – but this spike of value is countered by our global consciousness of our bounded condition and is degrading rapidly.

The image of our island Earth has taken a generation to kick in.

Humanities “production possibility” frontier depends on the resources available at any point in time, but the amount of accessible free energy is finite and bounded. Whether we are 1 billion or 10 billion.

The distance between the reality of physics and the reality of our economic and social structures are so great, that it’s hard to envisage any material solution.

When we look at facts, such as the fact we have lost four fifths of arctic ice volume since 1980, that cleantech is already a $trillion dollar industry (about 1% of planetery GDP), or that PWC think that there might be some kind of “business as usual” scenario in a 6C world  we know one thing: we have to change. Typically change doesn’t happen slowly: it waits a long time, then happens much faster than anyone expects. We need to remember that to create the problems of an industrialised world, we spent *multiples* (not fractions) of our GDP. While this has created many kinds of wealth, the systems-cost, the existential risks, are still struggling to be truly taken on-board.

I wonder, now, what change we will see in our generation, and if we will even be in a position to reflect on what was needed to make a meaningful difference.

I view environmental sustainability (including but not limited to climate change) as an existential risk. In the systems design of our economic, resource-scarce, finite and bounded ecosystem, there is a desperate need to create meaningful mechanisms to engage, at scale and in the mainstream, that enable people to discuss, understand and act on their environmental impact.

In an age of fiduciary, evidence-based decision-making, our balance-sheets are missing volumes of data.

We have tried to create laws, processes and standards (e.g. Kyoto, Climate Acts, ISO), and ratings (e.g. green scores) but none have managed yet to scale to hundreds of millions of people and businesses and dozens of countries in any meaningful way.

There are many, many reasons for this, but looking forward, we have new tools (the web, open data, new currencies, pervasive networks), and new ways to drive collaboration. In order to catalyse engagement, we can now create different starting points: the rest is down to collaborative (p2p) engagement between people with the absolute minimum of hassle (e.g. understanding methodologies, zero or low financial costs of change, and minimal time and effort) to improve our insight. We then need to automate everything, so the lowest barrier to entry is to do nothing at all (we’re all busy and/or lazy to change unless confronted).

One question is “how can we influence our Treasuries?”. I wonder who will be the first to truly bring change here – governments in the EU, China, or the USA? or Kickstarter and BitTorrent? Or who will be the first to join up our global data-ecologies to reveal the health benefits of energy efficiency, the true financial impacts of education, or the social benefits of codified law.

To catalyse change needs many forcing-functions: policy has a role and will play a greater role over time, but until then we will continue to rely on the goodwill and foresight of the small number of inventors, innovators, influencers that have actively engaged in trying to make a difference. We need to build more success stories, based on evidence, and redirect our collective energies at scale. And fast.

Essential reading
http://www.nickbostrom.com/existential/risks.html (you may find Existential Risks PDF easier to read)

Continue Reading »
Comments Off

I know it can be hard sometimes to grasp and translate ideas in a single-shot, over the phone, but this Guardian  “article” recounting their interview with me reads like it has been /auto-corrected/speech-to-text’d/ by an Engrish bot. Given it’s written in the first-person, I felt more than a little irritated by this.

So, in an effort to turn their semi-open-data into something you can read, I’ve turned the words in the article into something closer to what I actually said. I’ll try and keep as close to their format/structure as I can.

Where did your passion for open data come from?

I’ve always been involved in technology: writing software to create scientific models, analyse business data, or using the web create new services. At Jodrell Bank (the radio telescope), we were mapping large chunks of the Universe – helping to explore and understand its structure.

Throughout my career, from Jodrell Bank to AMEE (which calculates environmental impacts), I have been heavily involved in scientific modelling, data analytics and statistics (also now known as “data science”).

What their original article said:

“I’ve always been involved in technology-writing software and where that meets other disciplines, such as science and business. Working at Jodrell Bank, mapping large chunks of US feasability data – to take one extreme – was about tackling large questions around the structure and nature of the universe. In hindsight, that path – where technology meets larger questions about our surroundings – has been self-evident over my career; in Jodrell Bank, doing what would now be called data analytics, down to Avoiding Mass Extinctions Engine (AMEE), my startup company, which calculates environmental impacts.”

Comment: the idea of a radio telescope mapping “US feasibility” is quite entertaining. Never mind the spelling.

What will the Open Data Institute be doing?

We are facing some of the greatest challenges of our time. Whether social, environmental, or economic, I strongly believe that open data can act as a powerful tool to help us gain insight and act with confidence. For example, many large companies learned that simply by measuring their energy consumption (most were not), they can work out how to save substantial amounts of money and reduce their emissions. Open data can unlock insight and value.

The ODI is the first organisation focussing specifically on open data. Our vision is “Knowledge for everyone”. We are acting as a convening space for governments, companies, communities, and individuals who are already involved in this field.

Given the scale of opportunities, the ability to bring together different experts is important. We’ve seen over 700 people through our space in the last 3 months. These include NGO’s, Venture Capital firms, small business, artists, our own and international governments. To me this signifies that open data is a cultural phenomena.

I am very excited about the ODI’s potential to bring people together and catalyse innovation.

What their original article said:

“The ODI has the incredible position of being the first and only institute of its kind. We’re facing some substantial challenges – social, environmental and economic – and the institute will play a significant part in addressing those challenges, including bringing in efficiencies. For example, simply by measuring environmental impacts, 10%-20% of emissions can be saved by companies. That’s from having information in the first place – more information unlocks information.

National organisations and individuals are embracing more open data, and one potential for the institute is opening up knowledge for everyone, giving more insight across NGOs, capital venture companies, academics and so on. Cross perspectives and enabling people to learn is key. We’ve had in members of the Cabinet Office, small businesses, delegates from Asia and elsewhere – hundreds of people from all these different arenas, all interested in how we can move this agenda forward. Innovation and knowledge can affect social, environmental and economic issues – a triple bottom line, but also a very human side.

I see the Open Data Institute as an opportunity-capturing area, looking at how we can create something at scale, and I’m delighted to join in that.”

Open data – buzzword or game changer?

Open data is not new. The open web is the most successful information architecture in history, but it is not just a web of documents, it is a web of open data. In terms of timing, it feels like the mid-1990s – vast amounts of new information are about to be made available in a connected way, and the impact of this could be as game changing as the web has been over the past 20 years. This is particularly relevant as we increase our reliance on data to make decisions.

Working with world-leading experts, and the ODI’s co-founders, Sir Tim Berners-Lee and Professor Nigel Shadbolt, is an honour and their involvement signals the importance of our work.

What their original article said:

“It’s much like the web in the early 1990s. We’re at a tipping point where more and more information is available and so there’s more to unlock. I’m delighted to be working with some of the world leaders in this new movement, including Tim Berners-Lee and Sir Nigel Shadbolt. This is the most exciting thing I’ve done. Projects in the past have included building Virgin.net‘s early web presence, a pioneer in that space, a web-streaming business, and the last, helping to create an ecosystem around environmental data. But for me the opportunity is different. We’re in an age of data-driven decision making, with vast amounts of data. It’s huge and transformational.”

Comment: I’m not sure what the rest of my background has to do with the analysis of buzzwords. And they switched the Knighthood… (congrat’s Nigel!)

With a degree in astrophysics and a masters in electronic music, did you see yourself doing what you do now?

What I’m doing now didn’t exist when I did my degrees. In 1993, my first paid job was to review all music software on the internet. If you Google that now there are hundreds of thousands of results. I enjoy research and exploring new ideas, especially if you’re not sure what the questions are.

What their original article said:

What I’m doing now didn’t exist when I did my degree. In 1993, my first paid job was to review all music technology and, if you Googled that now, there would be hundreds of them. In hindsight, none of those things had really taken off. It’s very exciting when we don’t know what the questions are and we’re tasked with finding out what those are – and then looking for answers.

What’s your leadership style?

In order to create, we need to explore many ideas. Developing a culture where experimentation is encouraged is essential: we all need to be able fail, sometimes frequently, while we work things out.

The whole team should be involved in the design of the organisation: we are developing coaching and mentoring programmes for everyone. This will help create a strong organisation as well as help anyone in our team provide support to the companies we are incubating, and to the broader community.

One model I find interesting is the “servant-leader” model.

What their original article said:

A lot of what we do is through trial by example and error. That drives me, learning itself – and from those around me. We’re building team empowerment to help our teams make their own decisions and developing a mentoring scheme for our energy and emerging markets team. It’s about how we can best serve our teams and stakeholders. MIT is doing research on the idea of the servant leader, and that’s what I try to do.

Comment: Given leadership is the theme of their series, this was to me the most disappointing part to read

What three things would you take to a desert island?

A piano, an internet-connected laptop (if that’s allowed?) and a painting by my friend Ulyana. (http://www.ulyanagumeniuk.com).

What their original article said:

A piano, an internet-connected laptop (although I’m not sure that’s allowed) and a book or a painting by one of my friends, Ulyana Gumeniuk.

Comment: Ulya doesn’t write books

Continue Reading »
Comments Off

My slides from O’Reilly’s Strata conference “Making Data Work” today.

I described some of AMEE’s journey: through open data aggregation and distribution, accessibility, provenance,  and structure. But better data isn’t enough – no one (well, a few) really cares about the science or the technology. We need to engage with stakeholders to provide meaningful insight and relevance to their business. AMEE will be launching a new initiative this year to create an environmental score for every company in the UK.

Continue Reading »
Comments Off

During the Olympics Opening Ceremony, the creator of the web tweeted “This is for everyone” to millions of people around the world.

Decades since their invention, we are still discovering and unlocking value from the innovations catalysed by the open web, open internet, and open source. The Open Data Institute‘s mission is to demonstrate and unlock the value in Open Data.

Today, I am joining Sir Tim Berners-Lee and Professor Nigel Shadbolt at the ODI, as its CEO.

The ODI is a start-up – the first of its kind in the world. We have ambitious plans, and aim to have a substantial and positive impact for many, many people.

  • Incubate and catalyse innovative new companies
  • Help large and small companies develop and derive value from open data
  • Provide the right environment to inspire, train, and develop world-class talent
  • Enable organisations publish high-quality open data
  • Help shape standards in this emerging space

We have had fantastic support across the political spectrum, from academia, from the private sector, and from individuals.

Open Data creates the potential for anyone to innovate. The web was created using, and exists because of, open source and open data. I want to explore how we can best deliver;

  • data presented in a structured, “machine-readable” form so that data can be used by and between services (for example, using Apps)
  • data that is addressable via the internet and can therefore be linked together

I believe that

“information causes change, otherwise it’s not information”
James Burke, dconstruct 2012

There are massive benefits of getting this right. Governments, businesses, and individuals around the world are gradually coming to understand the power of data. The World Economic Forum has now categorised Personal Data as a new “Asset Class”:

“Personal data is the new oil of the Internet
and the new currency of the digital world.”

Meglena Kuneva, European Consumer Commissioner

And this is just the beginning: there is an emerging shift in our collective understanding of the power of connected, addressable information.

The ODI will help us reveal this power, guide us towards best practices, fair usage, and empower a new generation of innovators to create value – and in this definition I include economic, environmental, and social value.

What is Data?
This may seem like an obvious question, and to help anchor our language I want to be clear what this means. We live in an age where almost everything is, or will be, digitised. We are familiar with government spending data, health statistics, company financial reports, school assessments, and our own personal records. We are less familiar with data that is collected when we (as governments, businesses, or individuals) use the web, or devices that generate new data (such as location data from your mobile phone, or using Facebook).

I see two trends here: one is a growing set of opportunities for innovation – creating new services that improve our lives, the other is a growing sense of anxiety – that we are monitored and not in control of our information. I want to address both these areas.

What is Open Data?
Firstly Open Data does not mean “all data”, or that it’s a free-for-all. For example, your personal health data is extremely private. There are benefits, for example aggregated anonymous statistical analysis can help us make better decisions. There are also risks – we know that companies, governments, and individuals are not always as well equipped to handle information as we may want.

Examples (please send me more – I am keen to learn!)
- Public data released around MRSA has contributed to reducing death rates
- Company data released around environmental data has helped to catalyse the transition to  more energy efficient operations
- And even remarkable stories involving individual data could help to find new cures…

 

NB: I will remain on the board of AMEE.

Continue Reading »
1 Comment

This is good news:

“Nature Publishing Group’s (NPG) Linked Data Platform now hosts more than 270 million Resource Description Framework (RDF) statements….The data is now being updated in real-time and new triples are being dynamically added to the datasets as articles are published on nature.com.

Available at http://data.nature.com, the platform now contains bibliographic metadata for all NPG titles, including Scientific American back to 1845, and NPG’s academic journals published on behalf of our society partners. NPG’s Linked Data Platform now includes citation metadata for all published article references. The NPG subject ontology is also significantly expanded.

The new release expands the platform to include additional RDF statements of bibliographic, citation, data citation and ontology metadata, which are organised into 12 datasets – an increase from the 8 datasets previously available. Full snapshots of this data release are now available for download, either by individual dataset or as a complete package, for registered users at http://developers.nature.com.”

http://www.nature.com/press_releases/ldp.html

Continue Reading »
Comments Off

Energy Identity

July 19, 2012 by

Since AMEE’s inception in 2005, we have recognised that the emerging sets of data needed for carbon calculation and energy assessment present huge privacy issues.

Combined with the automation of data capture through smart-meters, mobiles, purchases and other “digital identity” sources there is a real need to address some fundamental issues.

As we help to glue together the instrumented world, what are the outcomes and what are the risks?

Energy Identity = The digital embodiment of
your physical consumption

(from slide 32 of my eTech presentation)

This concept applies to everything from individuals to businesses to countries, a product to a supply-chain, a home to a bank.

Issues include;

  1. Data ownership
  2. Data privacy
  3. Data portability (sharing) and control

The good news is that we’ve “seen this movie before”. In the 1990s we stumbled online, throwing our digital identity information all over the place, in an unstructured manner, and didn’t consider these points until it was too late. Initiatives such as OpenID and OAuth are only now trying to re-invent control mechanisms to address what we all need.

With energy, we have an opportunity to pre-emptively declare the rules of engagement. Some activity is already evident in this space (e.g. Google Powermeter testifing to congress). In the UK, since we have the UK Government as a client, I was able to seed some of these ideas some time ago (the UK is also gifted with the presence of MySociety).

To summarise, the issues include:

1. Data ownership

This should really default to you/your business (i.e. the source of the consumption).

The EULA of your service provider should ensure that you own your data and have expressly given permission to use it. Standard stuff really, but we’re a long way from that in this emerging dataverse.

From AMEE’s perspective, when we hold your data it’s subject to the EULA of the provider you are coming through (e.g. Dopplr) and defaults to you otherwise.

2. Data privacy

As with other services, the default should be to use a series of seperate silos.

AMEE holds each client’s data in separate silos (e.g. Google in one silo, Morgan Stanley in another). This allows for both digital separation and, if required, physical separation. AMEE can shard to enable this.

Further we anonymise the data on the way in – in fact we insist that clients don’t use AMEE to store e-mail addresses etc, and just use the anonymous key AMEE provides to link their user data. This key is held in their user database and points to the anonymized “AMEE Profile”. Given how much personal data is stored about businesses and individuals in AMEE we wanted to pre-emptively push away this risk, and instill confidence in our clients that even if AMEE were compromised, their users would remain anonymous.

3. Data portability (sharing) and control

Having established that ownership and privacy are the two foundation stones, we can then acknowledge that the ability to share information is extremely important. To do so opens a lot of issues, which we’ve been working on for a long time now, but we are confident that AMEE’s model enables extremely rich data portability without compromising ownership and privacy, by pushing control back to the data owners.

Thanks to effective anonymisation and security, we also believe that data mining and interpretation can be carried out without compromising privacy. Because AMEE has an effective security strategy in place, we can interpret and analyse the Energy Identities of, and on behalf of, our clients, and their clients, in an aggregate fashion, without becoming a “big green brother”.

The results of this research can be used to track the impact of policies regarding energy generation, distribution and use; and to confirm and develop carbon accounting protocols.

Summary

Thankfully most of the these issues are recognisable trends in the online development.

The challenge, and more importantly, the opportunity is to pre-emptively address these issues as we move to a deeper interconnected world.

The potential is for all of us to become involved in the development of our low-carbon economy, the democratization of energy and sustainability and, we hope, to avoid mass extinctions.

Continue Reading »
Comments Off

Things wrong with Companies House Crown Copyright data disclosure (which allows free copying)

Get the DVD rom of the data – SIGH

£30 for a copy of the data – PARTIAL WIN (it’s accessible)

£1200 if you want to actually save the data – FAIL

Crown Copyright Data is in a closed format -  FAIL (although in a trivially hackable form)

DVD self-destructs after 6 months – WTF!
(not stated at the point of sale, or on the phone when I called them)

Then it gets worse;

Windows *only* — not stated anywhere apart from the booklet inside the DVD (e.g. not on the website, phone or on the outside of the DVD)

Uses ActiveX wrapped into an executable — so I had to reset my default browser to IE…

Requires the DVD to be in the drive (“please insert DVD number XXX”) — so tied to that specific DVD

Has an “Award for excellence” badge on the back of the DVD.

My verdict: 1/10.  Not excellent. Not even good. I couldn’t do what I needed to with this open, Crown Copyright data. They have an XML API, but that is traffic-restricted.

FOI request sent.

Continue Reading »
Comments Off