Articles about rail travel in Europe often rhapsodise about the romance of the train, especially the night train – going to sleep in one city and waking in another. And when it comes to the environment, EU Transport Commissioner Adina Vălean described it as a “game changer” because it is “a truly sustainable form of passenger transport”.

The reality however, for cross-border travel, is a fragmented system, reeling from years of underinvestment and patchy cooperation between rail companies. And so environmentally unfriendly – but speedy – flights from budget airlines continue to be the preferred choice for most passengers.

“If somebody told me in 1995 that it’s going to take 30 years before you see significant change, I would have said, yep, well, I think I might do overseas development,” joked former British Labour Party leader Neil Kinnock, when he spoke to Investigate Europe about his time as EU commissioner for transport.

It was back then that the first of the EU’s railway packages – proposing the opening up of markets – was adopted. But nearly 30 years on and three more rail packages later (the fourth railway package came into force this year) we are far from making Europe’s railways a viable alternative to flying.

Introducing market forces

The first railway package sought to revitalise rail transport by introducing competition. Kinnock told us: “It was clear the existing models were simply leading to irreversible decline in both freight and passenger traffic. So, we had to find technical, legal and commercial means of restoring life.” One such means was to separate infrastructure like railway tracks from the operations – the actual rolling stock – to allow new rail companies to enter the market.

Some states implemented this fully. In other cases the management of the tracks and rolling stock were separated into different companies, but with the same holding company above the two.

But whichever way it was done, it introduced a track access charge for all trains that hadn’t been there before.

Journalist and transport expert Christian Wolmar describes this business model as “problematic”. Track access charges can cover not just the marginal cost for maintenance and wear and tear, but also a contribution to the capital cost of land, equipment, and so on, which means “you can never make it viable”, because “once you start charging and try to recoup your capital on every train that runs then you’re never going to make it cheap enough... The cost differences are just insane.” This charge either eats into the company profits, leads to higher fares for customers, or puts operators off running some routes in the first place.

This additional cost also sounded the death knell for many a night train. A map from the 1990s shows routes criss-crossing Europe, many of which are no longer running today. Rail enthusiast Mark Smith, better known as the Man in Seat 61, explained: “Sleeper trains were hit by the track access charge almost overnight out of nowhere for using track that is there anyway – and it has been cited as a factor in certain night trains being discontinued.”

Competition over cooperation

The rail packages were supposed to encourage cross-border routes by opening the market to competition, but they have often had the opposite effect as operators regard each other as threats to their business: “This whole ethos from the 1980s and 1990s and before, of cooperation within the Europe-wide network, has been replaced by a concept of every company running its own commercial operation for its own commercial benefit,” Smith explained.

Rail companies, for the most part, focus on their own countries, which is their core business. Christopher Irwin, from the European Passengers Federation, told us that the real problem with the packages was that they didn’t tackle the core problem, “which was the dominance of the national incumbents that don’t want to give up what they have invested in over the years”.

This lack of cooperation is best illustrated by trying to buy tickets for a cross-border trip that can involve dealing with different operators, using different websites in different languages. Even companies such as TrainLine and RailEurope can only offer a partial booking service as some train companies refuse to share their data.

Europe’s train companies are competing with each other when they should be cooperating. In the mid-1990s, the liberalisation of air transport set the stage for the biggest threat yet to their business. “Just at the time the packages were being developed, along comes this really big commercial competitor, the low cost airlines,” Irwin explained.

The airlines’ unfair advantages

Not only are airlines quicker and more flexible – routes can be changed to respond to demand while it takes years to build railways – they also have unfair tax advantages. One of the biggest is that fuel is exempt from VAT and excise duty for public transport flights. The Environment Agency estimates that the tax exemption for kerosene represents an annual tax loss of around €27bn for the EU. In the case of railways, both fuel and electricity are taxable. Air tickets within the EU are currently also exempt from VAT.

The EU is taking steps to address this, and has proposed ending aviation’s fuel tax privileges, but it does not go far enough according to Brussels-based NGO Transport & Environment, which reported: “The proposal still allows airlines to pay minimal tax rates for ten years before the proposed rates fully come into force, and the reforms only apply for fuel used on flights within Europe.”

It’s not just the airlines that rail has to compete with; in the last two decades, according to OECD data on infrastructure investments, there was almost €500bn more investment in road infrastructure than in rail across the EU, the UK, Norway and Switzerland.

The Swiss and Austrians get it right

But it’s not all gloom for the railways – the Austrians and Swiss provide good cross-border services. According to Smith, “The best example of a national operator that does actually believe in connecting Europe is ÖBB, the Austrian service, which also happens to believe in night trains.” ÖBB launched its NightJet trains with sleeper cars and couchettes purchased from Deutsche Bahn when the latter discontinued its night-train service.

How do the Austrians do it? This is a question that has to go unanswered as they have not provided information about their business model, but it is suspected that ÖBB is benefitting from a large amount of state support.

The Swiss heavily subsidise their railways and haven’t separated operations from infrastructure. Benedikt Weibel, former head of the Swiss Federal Railways SBB, explained: “I am deeply convinced that if you split up railway companies in different parts, like railway structure and other parts, you will completely fail.”

And Pedro Nuno Santos, the Portuguese transport minister, thinks that it’s time for a rethink of the EU’s rail policy: “Competition is important, but what we need to develop trains in Europe is public investment.”

In an unexpected twist, the EU may well want to look at what’s happening in the UK. The country that wholeheartedly embraced privatisation recently announced the creation of a new entity – GB Railways – that will bring infrastructure and operations back under one company. Christopher Irwin sees a lesson here for the rest of Europe: when things go wrong, “it’s time to move on to another formula”.