Placr News

August 26, 2010

Responses to Placr’s crowd-serving plan

Filed under: open data — Jonathan Raper @ 10:07 am

There’s been a whole conversation on Twitter about this proposal… here are some of the comments back and forth:

puntofisso Fri 20 Aug: I agree with this article> News on the tube API in @londondatastore and a proposal on ‘crowd-serving’ http://bit.ly/dnsUuF (via @MadProf)

paul_clarke Fri 20 Aug: RT @MadProf: Plan to crowd serve the tube API. Comments please on: http://bit.ly/dnsUuF .. fascinating approach!

londondatastore Mon 23 Aug: @MadProf interesting idea and look forward to response of the SME’s happy to point TfL in your direction if there is a head of steam

rollohome Mon 23 Aug: @MadProf sounds like an interesting approach to OD. I liked the SETI suggestion by 1 commentator: a ‘safe’ data set on which to experiment!

drewsonix Mon 23rd Aug: @MadProf Hi Jonathan – just read your posting re crowd serving. Wasn’t a big part of the problem the sheer size of data for all stations…

drewsonix Mon 23rd Aug: @MadProf ..and simply due to format? Couldn’t it be just “each train & its track locn” which we could crossref against cached tables?

drewsonix Mon 23rd Aug: @MadProf Something like Base64 representation of Set#, track loc, destcode, ismoving(y/n) for each train would mean a tiny fraction of data

MadProf Mon 23rd Aug: @drewsonix Data for all tube stations is not that large, nor too complex. But lots of users & some pulling whole feed at infeasible rate

MadProf Mon 23rd Aug: @drewsonix Challenge for tube API is finding a way to distribute data using open principles- hence crowdserving idea at http://bit.ly/dnsUuF

bensmithuk Mon 23rd Aug: @MadProf It’s an interesting idea, but given the loads it seems a shame TfL can’t just host and serve this properly from a CDN.

MadProf Mon 23rd Aug: @bensmithuk Absolutely agree, but meantime need a digital Dunkirk to rescue the feed! Rare opportunity for SMEs to get users & innovate

bensmithuk Mon 23rd Aug: @MadProf True, but isn’t complexity of proposed approach > than just proxying via Google AppEngine or Amazon?

MadProf Mon 23rd Aug: @bensmithuk Talking public sector procurement in a spending firestorm: not happening. If we take it over, its gets done fast, free & smart

bensmithuk Mon 23rd Aug: @MadProf :-) I didn’t mean them, I meant you / SMEs by collaborating.

MadProf Mon 23rd Aug: @bensmithuk Delighted if people join the party & pay to proxy a few million hits; assumed most would want to offer API calls to generate biz

countculture Mon 23rd Aug: @MadProf Like idea, but wonder if enough SME’s willing to do it; conversely if there were lots, whether it would be economic for each

countculture Mon 23rd Aug: @MadProf …Thus mean that they would drop out, and meaning only small number again…

countculture Mon 23rd Aug: @MadProf However, SME with business idea could always apply to join service, I suppose….

MadProf Mon 23rd Aug: @CountCulture Thanx 4 points on tube API crowdserving plan: idea is that in return for distributor status SME wud hav a chance to advertise

iapainter Mon 23rd Aug: @MadProf Looks great and a great idea. Nothing better than the buzz of a new startup despite its all consuming nature ;) Good luck #placr

daveaddey Mon 23rd Aug: @MadProf Problem is, SMEs (like ourselves) are hosting consumers, not hosting providers. We’re not set up to support high-load feeds.

daveaddey Mon 23rd Aug: @MadProf Also, demand for real-time transport data is inconsistent, eg v high demand during bad weather. Requires scalability we don’t have.

kemp_harper Mon 23rd Aug: Plan to crowd serve the tube API. Comments please on: http://bit.ly/dnsUuF

poggs Mon 23rd Aug: @MadProf Read that. In short, very fluffy and cute, ignores the fact we don’t know detail about what the issue is – bandwidth or API calls

poggs Tue 24th Aug: @MadProf I can’t see the business model in ‘reselling’ TfL’s data. Initial excitement will tail off; there’s only so much analysis we can do

MadProf Tue 24th Aug: @poggs what is biz model for any open data? No ‘white space’ coz no existing need. Need to make a market: transp apps sell by bucketload

poggs Tue 24th Aug: @MadProf I can only see two: a casual/simple “next trains” app on iOS/Android/web and a few people taking lots of data for detailed analysis

MadProf Tue 24th Aug: @poggs What bout live journey planners/ alerts/ oyster station footfall/ incident impact maps/ season ticket rebates/ station accessibility

poggs Tue 24th Aug: @MadProf Interesting uses, but I still can’t see more than a handful of people wanting to use the ‘big feed’ of raw data

MadProf Tue 24th Aug: @poggs Probably only a handful of SME tube API distributors needed. TfL will be able to it serve themselves eventually

The essence of these comments is that we shouldn’t have to do it ourselves but if we do do it then the business model for SMEs has to be clearer. I plan to post further on this… but the SME opportunity is to get users through their distribution of the TfL feed. Getting users is the hardest part of being a digital services SME, so if you get users but can’t monetise them then that is a more general challenge for open data. Persuading TfL to release data in this way… and then creating business for SMEs in London… would be a fantastic achievement for the open data movement.

Time is short… so let me have your offers to serve the tube feed if you can supply some bandwidth to join London’s great crowd-serving initiative!

Jonathan

August 20, 2010

Placr proposes tube API crowd serving solution

Filed under: open data — Jonathan Raper @ 3:10 pm

Currently, TfL have an in-house live tube departures server based on the Trackernet system. This server couldn’t support the volume of external demand when opened through the London DataStore in July and the service had to be taken offline when the load overwhelmed the server. TfL need an interim solution that takes a feed from the internal server and redistributes it.


There are various conceivable solutions to this problem if it is not done in house. For example, a commercial API service like Mashery can do it. And TfL have discussed Google taking the feed and redistributing it for ‘free’. The problem with commercial services are that they cost money directly; and taking up Google’s offer brings Google’s terms and conditions plus data formats into all business models built on top of the service.

We would like to see a solution that creates London jobs and showcases London technology while following open data principles. Why not simply allow some London SME’s access to the raw feed and allow them to cache and redistribute it as real time open feeds? This would be ‘crowd-serving’ open data and could use public domain licences like the MIT licence. The SMEs would innovate around the raw feed, building additional API calls which they could monetise any way they could. Since this would be an intrinsically competitive space, developers wanting access to the live data would have access to the open feed and the choice of suppliers for additional services.

How could this be done in practice? Technically, we would foresee TfL authorising selected SMEs to access the raw feed and pull the tube data into a web, database or memory cache. In practice, we think that updates any more frequent than every 15 seconds would be pointless as system latency and limits to train movement reporting make it unlikely that you could see change in the service at any higher rate. With the tube running 20 hours per day this generates c. 4800 updates per day into the cache. Each SME would then offer developer access to their 15 second cache using whatever system they saw fit to provide e.g. via

  • authenticated API key up to a limit of users to preserve service quality, or
  • an open API service in which demand would be limited by the physical limits on the server bandwidth
  • a Squid proxy to simply pass on the raw data.

If a developer found an open service or proxy to be too slow they would have to move to an authenticated service or moderate their usage.

The costs of providing one node of access to this kind of service are not huge. If a full ‘all stations’ read from the API is c. 200k uncompressed (we estimate), then getting this 4800 times per day requires nearly a gigabyte of data transfer per full user per day. A full user is equivalent in data transfer terms to about half a million users getting one station, once a day. Each SME in the scheme would have work out what sort of server package would provide processor, data transfer and bandwidth capacity sufficient to offer a distribution service. But we believe that the direct costs are affordable, in the order of £100 server costs per month to provide service to 50 full users… or 49 full users and half a million one hit end users. Of course each SME would also incur labour and overhead costs, but most would have the means to develop a freemium business model alongside their open feed.

The six million dollar question would be… who would qualifies to be a distributor? In our view the criteria should be that:

  • you meet the EU definition of an SME
  • you agree to offer the service for at least 6 months
  • you redistribute the raw feed no slower than the ?15 second update guideline, and
  • there are no access, charging or licencing constraints other than basic misuse terms and conditions.

TfL can probably handle 50 distributors pulling the raw feed on the 15 second interval base. If you met the basic criteria, participants could simply be the first 50 SMEs to apply. In time TfL could buy into the ‘best’ service or simply adopt the most efficient distribution method itself.

There are bound to be lots more questions, but the core principle of crowd-serving seems appropriate and suitable for one of the crown jewels of London’s open data. Let’s see if there are enough SME’s out there who can step up to the plate to offer an open feed and innovate around the data.

Jonathan Raper, http://www.placr.co.uk/

June 2, 2010

Navitime visit Placr

Filed under: Uncategorized — jonathan.raper @ 8:49 am

Placr were delighted to host a visit from Navitime last week. Navitime have been visiting the UK on a fact-finding mission, looking at pedestrian navigation and travel information solutions. Navitime offer the most advanced solutions in the world in this sector with their Japanese service hosting 4 million unique users.

The Navitime service is worth studying closely by those interested in the future trajectory of location-based services. The Navitime service works, and works well… I have used it in Tokyo myself. But it is based on wall to wall 3G services (GSM has been decommissioned in Japan), most users have flat data tariffs that do not punish data usage, and the majority of phones now have GPS supported by network-based location determination systems. Navitime have also invested in building imagery, sidewalk networks and real-time transport data feeds, much of which we do not have here.

Placr strongly believe in the future of LBS and have a number of propositions under development in this area. But LBS will likely only succeed when the location proposition is built on top of a rock solid user experience. Navitime have succeeded because they make excellent applications, but also because the mobile operators have built the infrastructure and ecosystem that supports the applications. European networks are improving (3G-only networks like ’3′ are building significant coverage), but expectations on LBS revenues in Europe should be scaled to the progress of the infrastructure and devices. Innovative design can only achieve so much without infrastructure: one poor experience of location determination which produces wrong or out of date information tells the user that a service is not ready for prime time.

Placr have developed significant technology in mobile positioning and travel information that we hope will help to join the dots between LBS infrastructure, data streams and applications. More on this soon.

Jonathan

April 9, 2010

Jowell response on DE bill (now Act)

Filed under: Comment — jonathan.raper @ 12:12 am

I, along with many others, wrote to my MP about the many problems with the DE bill, now scandalously passed into law as part of the washup before the end of this Parliament. The response (below) is a fence-sitting exercise suggesting that it is a finely balanced debate. Unfortunately, aside from the issues over disconnection that drew all the headlines, the real problem is way that legislators have tried to write todays’s law in the language of last year’s technology. This is doomed to failure in a very short time and will just have wasted everyone’s time while putting draconian measures on the statute book that we may all regret in a few years time when they get used oppressively.

Jonathan

Thank you for contacting me with regard to the Digital Economy Bill. I have received a large amount of correspondence on this issue and am fully aware of the strength of feeling on both sides of this debate. As you may know the majority of this bill was passed by the House of Commons last night and has now been sent to the House of Lords for further scrutiny.

I have written to the Department for Business Innovation and Skills on this matter and on a number of individual concerns raised by constituents. The response I received can be viewed on my website via the link below.

Response from BIS

Thank you for taking the time to raise this issue.

Regards and best wishes,

Tessa Jowell MP

March 31, 2010

Open Transport session at Where 2.0

Filed under: Uncategorized — david.mountain @ 8:32 pm

Looking forward to the Where 2.0 Birds of a feather session on Open Transport at Where 2.0 2010 (Blossom Hill rooms on the 3rd floor at 7.30pm).

Open transport data is a hot topic at the moment because governments and cities around the world are seeing the benefits of letting developers get access to real time transport data streams to produce mashups and mobile applications for travellers. So for example these cities already have policies in place:

San Francisco
Boston
London

The idea of the session tonight is to discuss how to encourage/ justify/ drive these movements and to find out what works. For example, we can already see apps like:

Baytripper
Traintimes

There are also some platforms being developed such as:

Opentripplanner
Georest

But it is not all plain sailing yet… there are issues around

  • Policies- what is the right way for administrations to release data?
  • Providers- who has actually got the data and through what channels do they release it?
  • Protocols- what types of protocols should be used to serve up data, and how should API’s be constructed?
  • Platforms- should the data be released as native/ hybrid or web apps
  • Come and join us for the discussion and bring your experiences!

    Jonathan Raper

    March 22, 2010

    Placr’s Response to OS Consultation

    Filed under: Uncategorized — david.mountain @ 1:54 pm

    Dear CLG,

    Thank you for an opportunity to comment on the proposed changes to the funding of the OS and the pricing of its products. This response is submitted by Placr Ltd. a micro company with 4 employees based in London that is in the business of B2C and B2B value added information services in the mobile and geospatial domain. Our business will be significantly affected whatever the outcome of this consultation is: we are responding to add our voice to others who want to see a change in the way the OS is structured and funded.

    Prof Jonathan Raper
    CEO

    Question 1:  What are your views or comments on the policy drivers for this consultation?

    Placr believe that geospatial data is the infrastructure of the 21st century and needs funding in the same way as other infrastructure i.e. from the public purse on the basis of positive cost-benefit analysis. Studies over 10 years e.g. 1999 Oxera study, 2008 Cambridge study have shown that the cost-benefit to the nation of funding OS is positive on a wide range of methodologies. Forcing OS to recover its costs has created an OS monopoly on value creation, distorted markets through tiered (Ramsay) pricing and led to wasteful internal government trading.

    Placr would advocate the government to fund the OS from general taxation to collect and release data at marginal distribution cost, and then to collect the tax revenue from commercial activity around its exploitation. We believe that the economic analysis in §1.24 to §1.28 does not take account of the new tax revenues from enterprises that could exploit OS data if it were released at marginal distribution cost. We are in a position to take advantage of such an opportunity and would be able to grow strongly on the release of new value added services.

    It seems to Placr that the consequent policy questions would then be:

    a) Can OS be innovative and efficient as a central government agency if it gets no signals directly from the market through revenues?

    Placr believe that the answer to this question is yes, as its mission would be to produce data with less ‘refinement’ than at present, therefore requiring less attention to the market for value addition. The release of ‘unrefined’ data such as large scale land cover, transport networks, boundaries, small scale maps, addresses and imagery at marginal cost would allow a utility maximising approach for society, as individuals, public bodies, social enterprises and corporations all innovate using the data. Much duplication could be removed from the OS catalogue eg 10K and 25K maps, Strategi and Meridian, Landform Profile and Panorama, Address Point and Code Point etc. as these can be derived from each other or from the large scale data in the database.

    b) How can the OS mission be directed if not by the market?

    Placr believes that data production should be tied to social and economic planning on long timescales linked to spatial data infrastructures such as the INSPIRE Directive and the Location Strategy (§4.38). The OS would be tasked with meeting update and completeness targets for the core unrefined datasets, and central government funding should be given in pursuit of these objectives. This would also require exposure of the real costs of the constituent businesses within OS as clearly most of the costs are in the large data collection from which the other datasets are derived.

    Question 2:  What are your views on how the market for geographic information has evolved recently and is likely to develop over the next 5-10 years?

    Placr believe that OS data is infrastructure and so its release should not be dependent on market trends in current GI products. We believe that tracking the market has distracted the OS from its core mission to secure and develop the infrastructure for the long term. The short term needs of the market drag the OS in different directions leading to frequent re-specifications of its mission. It is more important for infrastructure that the OS can ensure consistency in referencing systems e.g. across addressing systems, achieve convergence in parcel identification e.g. NLPG and Mastermap TOIDs, and that it meet the rising demands for GPS-consistent maps. Placr believes these are the more important long term tasks for the OS and that the market is not important to these.

    Question 3:  What are your views on the appropriate pricing model for Ordnance Survey products and services?

    Placr believes that the OS does not need a pricing model as it should be releasing a simpler and more unrefined set of datasets at marginal distribution cost. Attempting to manage price discrimination (§4.3) is doomed to failure as there is no natural foundation for segmenting the market, even if this approach did not monopolise value from the datasets, as it clearly does. Price discrimination is incompatible with a policy aim of utility maximisation.

    If the government decides not to move to the distribution of OS data freely at marginal distribution cost but opts to distribute it at true marginal cost  (although Placr would not favour this), then the true cost of each dataset should be exposed so that those who need the large scale data such as developers, planners and utilities should pay the costs of the large scale data collection. The marginal costs of deriving small scale data from the large scale data would be relatively low and these would make a lower call on the public purse in this scenario.

    Question 4:  What are your views and comments on public sector information regulation and policy, and the concepts of public task and good governance as they apply to Ordnance Survey?

    Placr believes that the ‘public task’ of the OS should be aligned to an infrastructure mission based on the free release of data at marginal distribution cost (see response to Question 2). This would avoid the need to wrestle with all the problems of the government acting in the market through the OS’s trading activities (§4.4). If this public task were set over the long term it would avoid commercial jeopardy for companies such as Placr as we would know that the government would not try to compete with us in selling value added products.

    Question 5:  What are your views on and comments on the products under consideration for release for free re-use and the rationale for their inclusion?

    Placr does not subscribe to the three options that this consultation set out and which define the scope for OS Free. Option 1 of ‘business as usual’ is incompatible with the Smarter Government initiative. Option 2 creates unnecessary complexity with DataCo and ProductCo entities. Option 2 should be a straightforward release of unrefined core datasets with a long term maintenance regime funded by the public purse (with tax benefits from the additional economic activity generating replacement revenue). In our view ProductCo should not exist in a OS institutional model involving the release of unrefined data at marginal distribution cost, as productisation should be carried out by enterprises like Placr.

    Question 6:  How much do you think government should commit to funding the free product set? How might this be achieved?

    The government should fund the release of unrefined data at marginal distribution cost from a new OS tasked with long term infrastructural development, as informed by legislation and advice from the Location Council and industry.

    Question 7:  What are your views on how free data from Ordnance Survey should be delivered?

    See answer to Question 5.

    Question 8:  What are your views on the impact Ordnance Survey Free will have on the market?

    See answer to Question 5.

    Question 9:  What are your comments on the proposal for a single National Address Register and suggestions for mechanisms to deliver it?

    This proposal is highly desirable. This objective should be part of the long term mission of a new OS that releases unrefined data at marginal distribution cost. A new OS operating on this basis would be able to take leadership of a national address database project working with local authorities, ONS and Royal Mail entirely within the public sector.

    Question 10: What are your views on the options outlined in this consultation?

    See answer to Question 5.

    Question 11:  For local authorities: What will be the balance of impact of these proposals on your costs and revenues?

    No response.

    Question 12:  Will these proposals have any impact on race, gender or disability equalities?

    In Placr’s view, they will not have any impact.

    March 18, 2010

    Placr supports WherecampEU

    Filed under: Events — jonathan.raper @ 12:25 am

    Great to see WhereCamp.EU was such a success this last weekend with a sell-out attendance and lots of great sessions. There have been hundreds of tweets and dozens of blogs, including the impressions of our own Harry Wood who was one of the organisers.

    With my City University Professor’s hat on Hanif Rahemtulla and I designed a questionnaire which was circulated at the meeting to discover more about open data developers. If you attended WhereCamp.EU and haven’t yet filled in and returned your survey form… please fresh copies can be supplied if you misplaced yours.

    Jonathan

    February 1, 2010

    Harry Wood joins Placr

    Filed under: Company news — jonathan.raper @ 4:57 pm

    We are delighted to welcome Harry Wood to Placr as a Senior Developer. Harry, lately of CloudMade has great experience of web technology, databases and open source mapping which will be invaluable to Placr’s development efforts. Harry continues to play an important role in OpenStreetMap developments, most recently in Brazil and following the Haiti earthquake. Harry will be working with us on API and business development. Welcome on board!

    January 29, 2010

    Placr play leading role in open data releases

    Filed under: Open Travel — jonathan.raper @ 10:00 am

    Phew! It’s been quite a month. As we develop business models for collecting and adding value to travel information, both local and central government have been busy releasing large quantities of public data through the London DataStore and data.gov.uk. Jonathan was heavily involved in the launch of the London DataStore as Chair of the event: you can see him introducing Boris Johnson and interviewing Sir Simon Milton in the coverage of the launch.

    Placr are very well positioned to exploit these developments as we have a great deal of experience in capturing, rationalising and analysing complex real time data sets. We are working on a number of significant opportunities in this area at the moment, and hope to open some services within a few weeks in association with OpenTravel. It’s going to be an interesting year!

    December 14, 2009

    Welcome to Placr

    Filed under: Company news — Jonathan Raper @ 3:42 pm

    After many months in ‘stealth mode’ while we developed the business model and put our financing in place, we are delighted to open Placr’s webpages to the outside world. We hope these pages will give you more information about the skills and the value we can bring to location: please get in touch if we can help you!

    Powered by WordPress