Category Archives: Web/Internet

Crowdsourced Flight Tracking

Earlier this week I had to head off to the airport to pick my wife up as she came back in on a transatlantic flight. As I’ve often done before I kept an eye on the flight using Flightradar24, and the Plane Finder app on my phone and the Apple TV.

Taking a look at the Flightradar24 site the Add Coverage link caught my eye – as I didn’t think there would be many people around with a spare radar station sitting around! However after a bit of reading it transpired that I didn’t need it.

Whilst in the past the online flight sites have been taking data from publicly available air traffic control feeds, now they are increasingly getting data picked up from ADS-B transmitters that aircraft increasingly use. Essentially each ADS-B equipped plane broadcasts a signal which encodes the location of the plane along with other details about their flight such as heading and altitude. The big advantage of the system is in areas that lack radar coverage improving safety, but since the signals broadcast on a similar frequency to DVB-T TV pictures it also means that a simple USB DVB-T receiver plugged into a home computer can be used to pick up the signals from the aircraft as well.

Given the really low cost of entry sites like FlightRadar24, PlaneFinder and FlightAware are significantly augmenting their data by offering their premium accounts for free to home users who supply data. You can use an existing PC, Mac or Linux box, but in order to get the account you need the computer to be running all the time, so a small low power computer like a Raspberry Pi is a much better option, and that is what is all three sites are suggesting. Whilst you could spend a large amount of money on fancy roof mounted aerials from my reading I figured that since we are located under the approach to Heathrow airport even the basic general purpose aerial that comes with the USB receiver would be enough to pick up a few planes. Additionally if I could get the Raspberry Pi feeding to all three sites – and there were plenty of people online saying that you could – I could get three premium accounts even for the potentially pretty small number of flights I could pick up.

So I drew up a shopping list.

First off, not having a USB DVB-T receiver I needed something to pick up the signals. I opted for the NooElec NESDR Mini 2 USB RTL-SDR & ADS-B Receiver Set which seemed to have pretty good reviews and which a number of people online were using – it also only cost £17.95 and included a basic aerial. I also needed a Raspberry Pi as although we have one, it’s part of the kids Kano so not something I can take over. I did look at whether to get the new Raspberry Pi 3, but since I didn’t need the extra speed or wifi I saved £6 and got the older Raspberry Pi 2 instead for £25.99. I also picked up a power supply, Raspberry Pi Case and memory card, and had all the bits for my DIY aircraft tracker for under £100.

Setting up the Raspberry Pi is pretty straightforward, you just need to grab a copy of the Raspbian operating system, install it onto the memory card and off you go. In fact you don’t even need a screen, keyboard and mouse if you have a bootable copy of the OS on the card as by default it has SSH access enabled so as long as it is plugged into a network once it is booted up you can access the Pi.

Once it was up and running, I initially opted for FlightRadar24, for no other reason as that was the site I’d initially read about feeding data from. That proved to be a bit of a mistake from the point of view of feeding to all three sites simultaneously. The basic idea is that you use the software from one site as the master feed, and then hook the other two bits of feeder software up to the third party data feed of the third. The trick seems to be to use the FlightAware software as the master feed and then tell the other two bits of software that they are talking to a receiver on port 30005 of localhost, then all three play nicely together. Once I’d swapped around and reinstalled the FlightAware software first I pretty soon had all three up and running and feeding data to their respective sites, alongside giving me information locally. If you start with the FlightAware software and a brand new SD card, you could also opt for their custom build of Raspbian with PiAware preinstalled which makes life a bit easier and is configured for optimal performance as a scanner.

There is a bit of variation between the software experiences. FlightAware is very much a command line setup, and connects to an account via username and password. Their local software is a simple page with a map of the planes and a list of their details. They have a much better experience on their main website, with probably the most detailed statistics on the performance of your receiver of the three sites. FlightRadar24 has a slightly better process, but still command line based, although when it is up and running the website is very basic just showing a list of the planes. You can however change the settings from the local website and restart without resorting to the command line. Plane Finder is by far the nicest local installation with an easy to use web based setup, and a nice local website that will give you a visualisation of the planes that your receiver is picking up, detailed log files, statistics of the performance of your receiver and the communication link with the internet, along with a web based settings page to reconfigure the software. Whilst all the sites give detailed step-by-step instructions that can guide a novice through, Plane Finder is by far the friendliest user experience.

So the big question is how well does it work?

IMG_5798I wasn’t expecting much from my little aerial sat on top of a filing cabinet in my office, but I was amazed. It’s certainly not picking up planes 200 miles away as a roof mounted aerial would, but it is picking up lots of flights within 50 miles, and one or two from a lot further away. It certainly picks up flights going in and out of Heathrow, especially when they’re coming in over Reading, along with quite a few flights over towards Gatwick and flights from further afield passing over the UK. In the forty-eight hours I’ve had it running it has picked up 2,859 distinct aircraft! If you compare my local radar view with the main site pages it’s clear I’m not getting everything, but I’m impressed by how many I’m picking up considering the cheap aerial and kit I’m using. Certainly if I wanted to spend a lot more money and get a proper roof mounted aerial I could probably track a load more.

So if you fancy a simple little project that’s not going to break the bank, and also want to contribute data to crowdsourced aircraft tracking I can certainly recommend building a Raspberry Pi based ADS-B tracker, and as a bonus you get access to all the fancy premium features on three of the main aircraft tracking sites. If you live in a populated area you’ll really only be adding resilience to the network, but if you’re in a more rural area there are definite gaps in coverage – there is a good map showing coverage on the FlightAware site – aside from that it’s also really quite fun if you’ve got kids to be able to point out a plane going overhead and say where the plane is travelling from or to.

Taxi

Recently I’ve been playing around with taxi apps on my phone.

For a while I’ve had Hailo on my iPhone for when I am in London, which is an app from a London startup that allows you to hail a black cab from your phone, but whilst that has expanded to other cities worldwide it doesn’t work in the UK anywhere outside the capital, so for day to day it’s not much use when I’m at home in the depths of Berkshire.

Since Hailo appeared, other competitors have turned up, the biggest and most notorious being Uber. The Moovit app that I have had on my phone for a while also for keeping track of buses and trains I’m using has had an Uber link up for a while, but again it hasn’t been that much use because Uber really only operated in the capital, however recently I noticed that rather than saying no cars available it would more frequently come up with availability for a car, particularly when I was in Reading. Whilst the company hadn’t officially expanded to Reading, they had expanded west along the M4 from London into Slough, Windsor and Maidenhead, and what I was seeing was cars who had carried passengers to Maidenhead becoming available when they had dropped their passengers in Reading and were heading back.

That caused me to take a look again and see whether any of the multitude of apps were usable for someone who didn’t live in the capital.

Hallo was definitely still only working with London black cabs, but GetTaxi, an Israeli startup with a similar concept to Hailo was now operating with the black cabs that operate in Reading. The only downside is that their service doesn’t operate outside Reading so whilst I could order a cab from Reading to home, I couldn’t request one in the reverse direction.

IMG_5766Uber would very occasionally offer me a ride even at home, but not often enough to rely on, so it looked like maybe there still wasn’t a viable option. However I came across another alternative thanks to a blogger I’d come across who worked as a minicab driver. There is lots of interesting stuff on his site, but the main point is that he lives in Brighton but operates mainly in London. He has worked for a variety of operators, including Uber, but is now working for large mini cab operator Addison Lee – the post where he discusses why he has gone to Addison Lee and is no longer working for Uber is well worth a read, but it also highlighted their investment in technology so I grabbed the app. I had seen Addison Lee cars operating around Reading so I knew they had expanded coverage to Berkshire – indeed since their sale to the Carlyle Group and the departure of their frequently controversial founder and CEO they are covering the whole country.

The app seems just as good as the equivalent apps for Uber, Hailo and GetTaxi, but unlike all of those it will offer me a taxi at my door, and allow me to book one at any time. Of course the on demand is not the five or ten minute wait you’d get in London – usually between thirty and sixty minutes at home – but booking an airport pick up or drop off the rates are comparable with any of the other cab firms I’ve used over the years, and they will also offer me a home to work, or work to home booking at a reasonable cost. The app also allows me to pay with Apple Pay or PayPal, and even retains the option to pay the driver cash (although one of the advantages of Hailo has always been that I never carry much cash these days, certainly not enough for a reasonable length cab journey). Certainly I’m going to give it a go next time I’m booking a cab, certainly can’t be as bad as some of the experiences we’ve had over the years.

IMG_5767As an experiment, having found a cab app that covered me at home, I then wondered how much their claim to cover the whole country really extended, and as yet, I haven’t found anywhere in the country it hasn’t offered me an estimate for an on demand request, or for a pre-booking, as long as one end of the journey or another is somewhere close to their main area in the south-east of England. This for example is a pickup request for the big hotel in the centre of Portree on the Isle of Skye, for which the app is quoting 295 minutes – whether they’d actually turn up if you made the on demand request is another matter as from experimentation 295 minutes seems to be some sort of maximum and is what it quotes in a number of places I’ve tried, but the app certainly suggests it will take a booking – an eye watering £1770 to come back home!

First Experiences with tadoº

We’ve joined the internet of things!

Old Heating Controller and ThermostatFor the past fifteen years, our heating has been run by much the same thing as most other people, a simple thermostat and programmer that gave us a limited ability to turn the heating and hot water off and on at set times during the week, and although we could vary the temperature in the house the heating would come on at whatever the temperature was set to.

tadoº Thermostat Showing TemperatureThis week the old thermostat and controller were taken out, and instead we had a tadoº system installed.

The kids like the illuminated temperature display, but it is doing rather more than just looking more high tech.

tadoº Extension BoxThe replacement thermostat and the replacement for the control box next to the hot water tank talk over a wireless link to a small box that is plugged into our internet router. On the tadoº website we can set the times to turn the heating and hot water off and on as we had done with the old programmer, however we can now set temperatures for individual periods, allowing us to set different temperatures for overnight and during the day for example.

However if it just replicated the functionality of the existing programmer and thermostat, aside from looking pretty and being able to set it up online it still wouldn’t add that much.

The main issue we’ve had with the traditional heating programming is that it doesn’t really fit with our pretty variable schedule, in order to accommodate the schedules for everybody during the week we end with up with a fairly generic schedule that ultimately results in the heating being on for a lot longer than it needs to be, and quite often the empty house will be being heated unnecessarily.

The other part of the tadoº system is a smartphone app that allows the system to tell if we are at home or not, and to not unnecessarily heat if nobody is home. Combined with the way the system tracks temperature changes and learns how the house heats up it should only be running the heating when needed.

So how has our first week been?

On the first morning the system turned on the heating two hours ahead of when we had set our wake up time, and therefore had the house warmer than it needed to be, however on the second day it learned from that and came on later. Having had a couple of really cold nights it has also turned the boiler on a couple of times during the night to prevent the overnight temperature falling too low. The presence detection has also worked well with the system turning the heating down for parts of the day when nobody was home, but also turning the heating back up again when they get home.

IMG_0398For techies like me you can also get loads more detail from the system of what the temperature is, and when the system is actually running the boiler – this is a graph of the temperature and boiler running from yesterday which was a day when Beth worked from home. You can see it running the boiler a few times in the early hours of the morning, then a big run to heat the house up for 6am (which it didn’t quite achieve) there is a shutdown when Beth took the kids to school, and a longer shutdown in the afternoon when she went out shopping and then to pick the kids up from school.

So on this day we saved a bit with those two periods when the heating wasn’t running, on another day when maybe we were out for longer the heating would be running for less time, we’ll have to see how this translates into our gas bill longer term.

Signing Up with the “Parasites”

So we’ve finally gone and done it, we’ve signed up with an agent to put our house on the market. It’s almost exactly three years since we pulled out of our last experience with the joys of the property market, so hopefully this time around will be less frustrating. We’ve had one hiccup so far which is that our local solicitor who spotted the issue over the parking at the Bewley Homes development last time has now retired so we’re scouting around for a new solicitor, however at this point we’ve been running the gauntlet of the local estate agents.

Talking to those who have moved recently we haven’t come across any ringing endorsements of any of the local agents, most people seem to regard dealing with them as a necessary evil. So we’re starting from a pretty blank sheet of paper.

My thoughts coming to selling the house have been that online exposure is paramount, so any local agent that is not putting properties on both the major property portals – Rightmove and Zoopla – is instantly off the list. Rightmove is the bigger of the two portals, however I’ve always much preferred the Zoopla site, in particular the level of detail you can get on a potential property, and the area.

I’ve also been taking a look at how each of the agents promote the houses on their books. So do they get the basic facts right? I can show you several examples of local agents who can’t get the basic information on Rightmove correct which means the properties they’re selling get categorised in the wrong place – four bedroom properties listed as three bedroom for example. Next what are the pictures of the property like – are they good pictures or do they look like quick snaps somebody has taken on their phone? Another local agent pads their adverts out with a whole load of pictures of the local area rather than the property they’re supposed to be selling. Also what is the floor plan like? There is quite some variation amongst the agents with pretty good quality plans from some, whilst another local agent has a number of properties where the measurements don’t add up and there are rooms with doors missing altogether, for example a bedroom that has no door at all, just walls. Then finally there is the wording on the advert, does it tell you about the property? Some agents do a traditional room by room tour, others a straightforward bullet point list, whilst another you get a little essay that doesn’t really tell you that much about the house. Even then are there basic factual errors in the text – I can show you several examples of local agents who get basic local details wrong, stating the house is in a different estate from where it actually is being a big one I’ve seen before.

Having gone through all of those we’ve had a couple of local agents out to value the house with a view to putting it on the market. Both valuations were around the same sort of ballpark, and both agents seem to be offering a similar service and both of them were looking to charge around the same sort of percentage commission which would work out as about £4000 to sell the house. The contracts are littered with the usual lock-ins, and clauses that result in the agents getting paid their commission if you trade your house in for a new build property as we’ve looked to doing before, plus the usual sole agent stuff.

However do you really need to spend £4000 to sell a house, or is the money partly going to pay for flashy cars painted in the estate agents colours, or a chain of high street offices that in some cases look like wine bars? Given that the primary way of looking for property these days seems to be online, do you really need to be funding a chain of high street offices anyway? I’ve been aware of eMoov, the biggest of the online estate agents for a while. They have a few houses up for sale locally, and I’ve actually had online chats with them on Twitter several times over the past few years. There was a good introduction to the company and how it was founded published last year in the Telegraph. More recently they have also done a more general overview of the online options, complete with a video with a traditional and an online estate agent discussing the relative merits of the two routes. The bottom line though is that the cheapest package eMoov offer is ten times less than the local agents were proposing to charge – is what they are offering seriously worth ten times more than going online, or is this another industry where online is shaking things up?

I did a bit more digging and there really isn’t any tangible evidence that the local agents can justify a tenfold price difference. Certainly with some of the online options you’re paying in advance, but if you’re serious about selling that doesn’t matter. If you’re not a serious seller then eMoov have a traditional pricing option too, but even that is a quarter of what the local agents were proposing. Indeed if you look at how the traditional estate agents are reacting, they’re basically name calling – in this article they refer to the online agents as parasites. They also seem to be reacting by setting up a protectionist market, as of the end of January they are launching their own property portal which has banned the online agents from posting, banned their own members from promoting on both Rightmove and Zoopla (participating agents have to pick one), and are encouraging their members to only post to the new portal first, holding back posting the new instructions to the better known portals like Rightmove and Zoopla. Essentially unable to compete with the online options they are trying to close them out of the market and giving their customers a worse service in the process.

Where does that leave us then? I have to say I was always leaning towards selling online – I bank online, buy groceries online, do pretty well everything else online, and I can find no overwhelming evidence that the local agents are doing anything to justify charging ten times as much as their online competition. The real clincher though was that the local agents are resorting to setting up a protectionist market and calling the online sellers “parasites” rather than stepping up and actually competing, so last night we set the process in motion, and had confirmation via e-mail first thing this morning. I’ll post more as we work through the process.

UK Wide Public Transport – What About Google Maps or Moovit?

When I published my post about whether anyone could bring the level of flair Citymapper has shown in London to UK wide travel I not surprisingly got some comments back about services I had missed. To be fair I deliberately didn’t list off all the many public transit apps, as there are absolutely loads out there, however I thought I’d discuss two which came up, Google Maps and Moovit.

If you believe the Daily Mail Google Maps now has real time data for the whole of the UK, unfortunately like a lot of articles that appear in the paper, or on their website that is total rubbish. If you read the actual Google Press Release what the said is this:

We’ve added every single transit route in Great Britain to Google Maps—making it easier to get anywhere from Land’s End to John O’Groats.

They then say that they’ve added real-time information for Vancouver and Chicago, they don’t say anywhere that they have added real-time information for the UK. If you click through to the post by their GB product manager this explains that they’ve taken schedule information from National Express and Traveline so essentially what they’ve added is a much nicer interface to the existing UK wide travel planners. Basically somebody at the Daily Mail can’t actually read a press release properly, and certainly didn’t bother to check the actual site to see what is going on.

Screenshot 2014-07-05 22.19.27Doing what the Daily Mail didn’t bother to do taking a look at Waterloo Station in Google Maps, for the Underground station they list the scheduled times of the trains, which might be a bit of surprise to most travellers on the tube, that there is actually a timetable!

Screenshot 2014-07-05 22.19.00Swapping over to Citymapper we have the actual departures in real-time, as they have for stations all across London.

Screenshot 2014-07-05 21.59.45Things are even more sparse when it comes to National Rail – although route planning from Waterloo will bring up accurate times, there are no departures on the Google site.

Screenshot 2014-07-05 21.57.26Taking a look at the same station in Citymapper you get a complete list of realtime departures. Citymapper being London focused has live departures for key commuter routes outside the capital, for example they have departures for Reading station.

The situation is much the same when you look at the smartphone applications – no real-time data on Google, loads within London for Citymapper.

IMG_4133Moving on to Moovit, once again the issue is real-time data. They are a bit better than Google, in that they do have some real-time data, specifically for some buses, but once again they don’t have the real-time data fields for National Rail. Looking at Reading this time, this is an example of what Moovit might show – note the message at the bottom mentioning that real-time departures will be highlighted in Orange.

IMG_4132This iPhone screenshot is from the app of an excellent site called Realtime Trains that is a trainspotters paradise in terms of the , so that app is telling me platform numbers, whether a train is running late, and interestingly has a different departure time than Moovit is presenting.

That also highlights another problem, the quality of the data in Moovit. One of the app store reviews comments that all the bus times on the reviewers local route were ten minutes out. It’s also remarkably difficult to find London Euston station on Moovit – it has Euston Square station and the Underground station, but the only way to actually get to see the scheduled departures from Euston using the app is to find the station on the map and click on it – it doesn’t come up in searches at all. Reading departures can equally be problematic when searching by hand as the app retains the historic split between the old Reading Southern Region station and the Reading Great Western station even though physically they’ve long since become one station. This means that there are two icons for Reading station almost on top of each other and two separate lists of departures.

I have to say, I really want to like Moovit – as an avid user of Waze crowdsourcing public transport data seems a great idea. The developers seem to have come up with a way of tracking journeys that doesn’t decimate your battery level, but without accurate schedules and using all the available real-time data that is available for buses, trains and the Underground it is pretty useless, where Citymapper excels is that it tells me what is actually happening, not what should be happening. There is also a split between Scotland and England which none of the other route planning apps have which makes it impossible to plan a trip that goes over the border – whilst it may be argued that they are splitting by country, the actual underlying data they’re using for the schedules is UK wide resulting in the trains for Scotland apparently only being available on the English area, so it seems a slightly arbitrary split that renders Moovit even more problematic and unnecessarily so.

So my initial conclusion remains, even taking into account Google Maps and Moovit, there is still no UK wide multi-modal public transport application with the flair of Citymapper – who is going to produce one? Will Google introduce real-time data? Will Moovit sort out their schedule data and add in more real-time? Or will Citymapper expand out to cover the whole of the UK?

Can Anyone Bring Citymapper Flair to UK Wide Travel?

Screenshot 2014-06-29 21.38.12I’m a real fan of Citymapper – for travelling around London it does a great job integrating all the trains, buses and the underground together in an easy to use app, a trick they’re repeating in other major cities across the world.

However there is one big issue, whilst I go into London, I don’t live in London, I live out in Berkshire. The app includes train services out as far as Reading, but if I want to take a bus to the station I need a different app like one of the Bus Expert apps to get me to the station. The only buses in Citymapper are those run by Transport for London, so even connections for the outer reaches of the London Underground aren’t included for where I grew up around Rickmansworth.

That’s not to say that there isn’t the real time data around, Citymapper make use of data feeds from Transport API, and their data covers a much wider area than just London, indeed you can find example sites and example applications that use their data on a wider level.

There is also the government backed Traveline websites which have a set of regional travel planning applications, each working and looking different and having been developed by different suppliers appointed by the various regions, and then Transport Direct offer a national route planning service, along with a mobile web site, but no equivalent app. All of the government apps and sites work, but they aren’t exactly the kind of modern web and app experience you get from an app like Citymapper.

Screenshot 2014-06-29 21.36.38One of the better UK wide route planning apps is a cut down version of the Rome2Rio app called UK Transport Search, it’s got a small footprint and is pretty quick to produce reasonable routes, however it lacks any of the realtime information that Citymapper has.

You can certainly plan trips and navigate around the country using a combination of apps, but nobody seems to have produced the kind of multi-modal public transport application that Citymapper have produced for London on a wider scale. We have all the bits, but nobody has actually pulled the whole thing together, so who is actually going to do it?

Can You Trust User Generated Mapping?

A couple of days ago The Next Web ran an interesting and detailed article about the rise of Open Street Map and their ongoing attempt to compete with Google Maps as the go to mapping solution.

Open Street Map data has achieved several notable successes as companies have switched from using Google data to theirs, but the critical question is whether the data is any good?

I have a bit of a love/hate relationship with user generated mapping solutions. As you will see elsewhere on my blog I’m running Open Street Map based maps on my handheld GPS, and my satnav of choice is Waze, which although it doesn’t use Open Street Map, uses maps made in a similar way with user generated mapping.

The strength of both platforms is that anybody can edit the map, so problems get solved a lot quicker than errors in the major suppliers maps get sorted, but that is also their biggest weakness.

IMG_3839I’ll give an example, a couple of weeks ago I was driving to work, and I came across this on the map just south of Eversley, a route I drive most days.

To explain what you’re looking at, the map was showing that the roundabout where the B3272 splits from the A327 was about five times as big as it actually is, and was a residential street rather than a main highway.

Getting home and taking a look on the Waze map editor confirmed there was a problem with the map data.

IMG_3841What you can see on here is the editor view overlaid on a satellite view of the same junction. The roundabout is highlighted in red which indicates that there is missing location data for the road. Checking the road elements the new roundabout had been put in place by a new user who had done little editing before, and who had obviously accidentally damaged the map at that point, messed it up and left it. The problem of course being that nobody else spotted it until it was too late and the update went out to all the users.

To the credit of the community shortly after this screenshot was taken one of the higher level editors sorted the problem as needless to say it got reported fairly quickly, and unlike errors in TomTom or Garmin maps the fix appeared on my copy of Waze within three or four days.

Screen Shot 2014-03-01 at 22.50.53Of course the professional mapping firms don’t get it right. For example this is the dual carriageway that Google Maps has on the B3272 in the middle of Eversley Cross – if you live near Eversley you’ll be forgiven for having missed it previously. What they have now is a bit of an improvement over what they had originally where the westbound carriageway followed the footpath across the front of the houses fronting on to the green.

Screen Shot 2014-03-01 at 22.48.50There is also this long standing problem in Camberley on the TomTom maps. Again it’s been improved over time but it still incorrectly labels Charles Street as New Southern Road, and still has two connections at the northern end of Southern Road that just don’t exist.

Screenshot 2014-03-01 23.00.26In both of those cases, Waze and OpenStreetMap are right, this is the Eversley Cross dual carriageway on OpenStreetMap.

So can you trust User Generated Mapping?

I’d say that you can trust it to the same extent as “professional” mapping – both have errors and problems and you need to be aware of that and use your common sense using them as you should with any map, even one with the long established reputation of the paper Ordnance Survey maps. The key advantages that user generated mapping can bring is that if there are mistakes they get sorted quickly, and new roads or changed layouts will often appear in Waze or OpenStreetMap long before they turn up in one of the periodic updates for a regular satnav.

Really the comparison of Wikipedia with OpenStreetMap is a good one, both are excellent sources of information, as long as you are aware that they might be wrong…