A couple of weeks ago I wrote about how some Star Alliance carriers are attempting to restrict websites scraping award availability (this started with ExpertFlyer announcing it would no longer show Star Alliance award availability). There’s now a major update, as the first lawsuit has been filed regarding this…
Continue reading: Airlines Try To Shut Down Websites Scraping Award Seats
Share your questions, experiences, and thoughts below.
The issue is not mainly websites scraping award inventory, but mileage brokers, controlling tons of Super Elite 100K accounts, squat, transfer award tickets like no one else matter.
Just by adding a random delay before cancelled award ticket back into inventory, and add a cancellation fee for everyone like 7 days before departure, would solve a lot of those mileage brokers.
No snark intended... but don't you have a financial interest and partnership with a service doing exactly this?
I agree, I'm not sure how effective banning public/"commercial" scrapers are. Any broker of any decent size will likely want to roll their own scrapers anyways. The only real way to combat fraudsters would be some sort of verification system (tied to a real identity) for FFP accounts, specifically targeting those with suspicious activity.
Everything else is just whack-a-mole. Ban 1 account and 2 more pop up.
Scraping yes, not milage brokering. Regardless, I like how he didn't use his business as the example. That was my first thought as well though.
Aeroplan's actions feel incredibly disingenous when they're not taking clear and obvious actions that would directly target mileage brokers. If that's truly the problem then limit who we can redeem for (a nominated list of friends and family, for example), but don't take drastic action that affects everybody (pulling many different types of awards entirely) and blame it on the services that most people are using to get around how stingy award availability is these days.
I for one hope the airlines find a way to shut down those brokers who are using all the practices in your article on broker fraud methods.
It’s going to be a fine line that the airlines walk to stop their frequent flyer programs from being abused. There are definitely ways of doing it by tightening up the rules on who can book a flight and close in cancellation fees. However they are not going to want to jeopardise their lucrative points and miles selling divisions.
Clearly seats.aero was the most effective of the award search sites. I’m going to try and book flights to use most of my points very soon. If seats.aero gets shut down it’s basically game over for my aeroplan miles.
Hi Ecco,
honest question because I am not with Aeroplan (but Miles&More): why do you guys need seats.aero to book awards? Is the Aeroplan search soo bad?
It sounds like all your Aeroplan miles are now worthless as seats.aero does not work anymore? What’s the relationship between those two?
The solution is easy: Provide reasonable search tools and availability alarms for your own customers. This way no external tools are necessary. If its about the server load: provide APIs for external searches. Easier and more efficient.
This is a good, and fair, overview and kudos for not speculating on that legal issue beyond your actual knowledge.
One additional factor is that any business is concerned with customer engagement though their website and when a measurable percentage of the interactions move to other parties it probably rings alarm bells.
The problem is that finding points seats on airline sites is generally a horrid experience that only lets you search for point to point redemptions.
If the airlines would enable 'show me everything' options then they could block all of these paywalled search sites.
Could be easily circumvented by the airlines if they'd
1) only allow account owners to book awards for themselves and persons they have previously booked/traveled together with.
2) Randomize inventory, giving every frequent flyer a slightly different award availability.
Who are the major mileage brokers? Where do they find these Super Elite accounts?
Reading on FT looks like one of ACs points via Nasr is partner IT considerations. So for example maybe AC is getting in hot water with an alliance partner because its own site is making too many calls to the partner system thanks to the scrapers and the weak partner(s) aren’t capable of handling the concentrated load.
He's being dramatic. I have millions of AP and don't need tools. It'll be rougher obviously, but if you know the routes and airlines and patterns you can significantly reduce your time spent. Yes, I'll prepare more for future bookings and more time invested, but I'll still get EXACTLY what I want.
No relationship with AC. Same proposition as it was all the years before seats.aero
@Ben, what is the "saver award" level you often refer to in articles regarding Aeroplan (and maybe others)? It's not an Aeroplan term I'm familiar with.
I think it's a great idea and only fair. I don't subscribe to any of these brokers. I search for awards on my own manually, the old fashioned way. Been doing it for decades. Lately, it's been extremely difficult to find the award flights that I want... even a year out. I don't know if it's because there are more people playing this miles game or if it's because of the automated scrapers that some of brokers have. I think it will even out the playing field if the brokers are prevented from using their automated software.
When airlines make award seats available, that means they are okay taking that booking. This isn’t a way to reduce award bookings for consumers and I’m surprised at how many people are suggesting that.
The main problem with AC award searches is you can't search awards for flexible days. AC has so many partners beyond star alliance so, while you can search United website for Saver Star Alliance awards and then search the exact dates found there on AC, the non star alliance awards you have to search day by day. But I think its quite a bit of hyperbole to suggest it's game over for Aeroplan miles. Unless you just refuse to do any work yourself to find award flights.
Agree with this. The situation has become like Ticketmaster scalper bots causing issues for regular consumers
Pretty sure it's a game of cat and mouse. Airlines are gonna shut down scrapers. Brokers have to get more clever. To fight brokers they just need more rules. Pretty sure these airlines can just run a query to see which accounts have activities that are suspicious. Maybe it's a limit on how often you can cancel tickets last minute and if you do that, they can restrict some aspects of the account. It's like ticket master. If they forced you to upload a photo ID for purchase of the concerts most likely scalped to young fans, these scalpers can't exist. Make it onerous to book because it already is for us normal ppl who redeem once or twice a year. Airlines want to increase their take with credit card companies and you can't cater to brokers or even users of scraper sites bc those are the minority. Most of these CC sign-up ppl are plebians. Make it work for us.
Star Alliamce are in the wrong here, they aren't tackling the right problem with this legal strategy. Part of the reason elite members use third-party tools is due to lack of usability on their own search portals, and difficulty finding upgrade availability (such as PlusPoints).
I am a subscriber and supporter of seats.aero, and a frequent flyer with top status on star alliance.
Aeroplan / United / star alliance should focus on adding friction for points brokers instead. For example, accounts that routinely book tickets for new customers that never flew the airline before, with high volume of transfers from different sources, different credit cards, etc.
The answer is not to make point redemption more difficult for frequent flyers, but to deploy smarter IT systems that combat fraud and point brokers.
This isn't some sort of complex self-learning AI system - you look at patterns and build some basic rules to stop brokers from booking random tickets over and over for passengers that don't even fly with the airline.
For example, British airways allows you to have 6 nominees for your "friends and family" list to avoid abuse of selling your points to someone. The list can only be changed every few months. That is a far better model than making it hard to find award availability through great tools that automate alerting and help you search across airports and programs.
Pretty obvious that Air Canada printed too many miles for too low a price and now want to inhibit searching for the most valuable (and expensive for their program) partner saver awards.
They've been blocking availability on many airlines for months.
Seems the airlines are missing out in a potential added revenue here by selling access tokens to their apis and limit the number of hits (similar to Google flights API).
If airlines made their award availability more transparent and search functions as good as these 'scrapper websites' people wouldn't feel the need to pay for these services. Because airline award search engines are so crappy and fraught with too many errors people look elsewhere to look for availability. Why can't airlines build alert function so that people wanting to travel can book award tickets? Because airlines don't want customers to know that there is free(award) seats available. They want people to keep spending money on them
It all depends what the airlines hope to do with the space, are they going to try to sell that space or offer elite upgrades? Why sell the ability to other companies when they should get 100% of the profits?
I worked for Worldspan when Orbitz was being created. Orbitz was an attempt to sell remaining seat inventory and was a cooperative effort by airlines to make that happen. The huge amount of data we held was staggering, why would any airline sell that off for API access fees? They wouldn't and won't.
This might seem anti consumer but in the end it makes it harder for the average person to find space, that only increases my odds of finding what I want!
I agree. This has made award seats much harder for the average buyer to find. Other pet peeve is folks making speculative bookings only to cancel. I prefer the old fashioned way to find them manually.
1. Yes - rewards should not be transferrable - should only be redeemable by the member and others travelling with account holder need to be on same PNR.
ANA has a great solution for mileage brokers. They require passengers to register up to 10 nominated people who can receive mileage awards from their account. If you want to change a registered user, you've got to fork over 5,000 miles.
Who cares about the average person? If they are not using the tools available to them, they are suckers. The customers using these tools are the most engaged customers - the ones that deserve the award seats.
Even easier solution: Limiting award bookings to the account holder himself and other passengers he has previously traveled together with on the same booking.
2) would make the scrapping pointless as the scrapper tools would no longer be able to find the actual award availability for that member.
Agree that something has to be done. However I fear this will go the way of concert tickets where published rates mean nothing. If airlines are smart (and most are) they would follow the concerts/sports/ events model and it will be much worst than what we have now. It will become a zero sum game so enjoy it while we can.
Google Flights APIs don't do anything with the inventory .. they just sell access to the information and scale the cost based on the number of hits. That's what i'm suggesting. It's better than just slamming the door on people willing to pay for the information or ease of search.
Curious what are the hard stats re award inventories, vs previous years etc. ? This would answer a lot .
"the volume of these searches puts burdens on Aeroplan and partner IT systems, and enables points brokers."
I buy the second rationale but the first seems spurious.
Also, it's hard (not impossible, but hard) to see how the airlines prevail here without the ruling also crippling the AI industry, which relies on public information in a similar fashion, making me even more skeptical that their objections are on solid legal ground.
Max: So I first need to book a revenue flight for my son and me before we could fly on an award ticket?
How about my expat colleagues who wants to fly in his parents on an award ticket?
@Ryan - If the average person perceives the system is rigged, they will abandon loyalty incentives and move on. And this population is probably not an insignificant number so the airlines are protecting their flanks.
I bet brokers just make new accounts and fund it, and then make new ones and fund those, forever and ever. It's a hard problem.
You need to give seats.aero a go to see its functionality in relation to aeroplan. I don’t have the time to go searching for unicorns, as some do. Also all the airlines have different award availability even if they are in the same alliance so searching united is not the answer. Also ringing aeroplan to book is not viable unless you have hours to spare whilst they put you on hold.
I’ll definitely be on seats.aero this weekend!!
The lack of availability is a function of the regions you are trying to travel from too. There is generally more availability for Oceania to Asia where I am based. Anything long haul to and from the US is very hard to get. This is understandable considering the number of points currency being printed by the US credit card companies. I’m basing this on experience of trying to book a rtw trip from Oceania.
I think this is definitely part of it and also the reason AA have stopped having points sales. Too big a liability on their balance sheet as well when they print miles.
Maybe Aeroplan’s partners are getting sick of all the cheap Aeroplan award people cluttering their premium cabins too.
The brokers are definitely a factor too but there are definite ways to shut their business model down if the airlines really want to. It’s just how to do this without affecting their private ffs.
If they can make their services easier to use, we wouldnt need to use services like seats aero. Its been ridiculous how hard it is to book anything with points these days. They want to maximize their profits i get it. But dont screw your loyal customers over.
The airlines make it difficult for folks to find those choice awards because they will earn more money if the consumer pays more or never uses his airline miles. British Airways is one of the guilty parties in which you can only move from one day to the next to view their awards. Even if you call them up, the agents have to laboriously go through day by day to see if award space if available. So there is no upside, in their view, by making it easier to book cheap award tickets. The point broker argument doesn't pass the smell test.
I love that you used a picture of Lufthansa's first class to illustrate this post! It's like a five-star hotel in the sky.
So having millions of people constantly searching for award seat give a relief to their IT infrastructure as opposed to a dozen “scrapers”? If I search on seataero then I don’t tax any air Canada or United IT resources.
Nasr is and always had been a lying sack of sh!t
AC is blocking partner awards in favour of their own 400k metal awards. They also minted too much mileage and brokers are running wild with bought miles and family pooled accounts.
But you gotta blame someone/something.
B
Anyone with a brain can see that this is exactly what's happening.
The scraper/fraud "issue" is a deflection.
Well if airlines go the way of UA, they simply eliminate all saver awards and gouge their customers. OR virgin, allowing two awards per plane. Or just throw the whole award scheme in the trash, which seems closer than ever. I wonder how the banks feel about this, since they make money off of points cards.
Somehow I suspect the LH group in this. I have had EF alerts for six months on LX, LH, and OS. Not one saver award has popped up for Europe to EWR. Coincidence?
Yes, I subscribe to ExpertFlyer but completely fine with killing all these leech tools.
It’s quite easy if airlines were so inclined to solve this.
1. Limit redemptions for every account to just 10 people max.
2. For every account sign up, ID verification is needed (e.g. Social Security Number) to prevent brokers from simply signing up for new dummy accounts, and to make sure that the same ID isn’t used to open multiple accounts.
3. Points shouldn’t be transferable between other accounts (e.g. member A on Aeroplan shouldn’t be able to transfer points to member B on Aeroplan).
Why not just provide an API for these searches, then there won't be any problems with technology.
They already do this for travel aggregation and regular flights...
Agreed. If fraud and brokering is a problem then address those. Not something the community values and a small fraction of the time is brokers. Why soend hours on the AC website to find out they're blocking Eva, ANA, Thai, etc. when seats.aero will tell you in 5 minutes. No sense of urgency is fixing the blockage of those carriers.
I posted this on Live and Let's but I'll also post it here.
If they're serious about cracking down on fraud and brokering then go after those things. Dont go after a tool that isnt hurting anyone and that the vast majoroty of the time is the community, not brokers. This will not have a significant effect on fraudulent brokers for 2 reasons. 1. These brokers likely could build their own scrapping tool for the airlines and routes they frequently use. 2. They could use sweatshops in Bangladesh, Laos, or Cambodia and have 10 yo's just manually searching day by day on the AC/UA website. Remember, fraudulent brokering operations are run primarily out of PRC(mainland China) and India, respectively. In order to seriously crack down on fraud and brokering, North American carriers need to add a designee list as others have mentioned. You could add 2-6 people to this list with possible exceptions for immediate(parents, spouse, and children) family members and there will restrictions on how often you can change this list. Additionally, adding protection from bots that snipe seats the moment the calendar opens would be good as well. Remember different people play the points and miles game differently, some of us are economy vs luxury, close in vs year out, point to point vs complex multi legs, visiting friends family vs aspirational vacations. Different people do things differently and as long as people stay within the rules there shouldnt be issues. They should go after actually fraud and brokering being committed instead of after tools that make this hobby, slightly easier. Alternatively, just make the search better. Also, AC is so concerned about a community tool that makes things easier, yet they have no sense of urgency to stop blocking partners like Eva, Ana, Thai etc. Why spend hours on their site searching for partners they're blocking when you can find out in 5 mins on seats.aero? Like come on.
There's a huge difference between hitting an API and sending just the requested data vs serving all of the data on the pages each time there's a hit to retrieve it. Not to mention having to hit multiple pages to drill down for the data vs just the API hit.
I could accept this as a temporary measure. But the airlines really need to find a way to go after and close down the accounts used by the point brokers. In the meantime, this feels like class detention. We’re all being punished for the transgressions of a few.
I watched that facebook interview with Nasr. Thought he came across as very genuine and trying to find solutions for a very complex IT situation. Why do you say he's a 'lying sack of sh*t'? Their program has been revolutionary and they've found they need to make changes.
It's a shame if the brokers are using these scraping sites to perpetrate fraud and game the system. I'd definitely be acting if I were them in this case especially if it's ultimately affecting their bottom line.
It would probably be good if the whole issue of website scraping got tested in court once and for all as to whether this is a legal practice. Of course, as others have mentioned there are plenty of ways that the brokers can be stymied with a few rules put in place about opening accounts, who you can book for etc. Disclosure I personally use seats.aero a lot and hope it does continue!
If airlines made award seats reasonably available, they would not have the "scraping problem" in the first place...
While I don't like it, learning from East Asia carriers is an easy way to put a brake on the crazy mileage broker scene.
The problem is, you’ve got phantom availability, and you’re missing a ton of airlines.
If you take away the scraping, ok sure MAYBE I can get a better redemption at the end of the day. I’m willing to buy that excuse.
But if I can only get like 3 of the airlines in the program, what good is it? Fix that first.
I'm just playing the game by the rules set by the Airlines. Their business model is to give away points like candy and make it difficult to fulfill their promises. My job is to play by the rules and maximize free perks. And if that means using seats.aero then so be it. And if it means canceling and then rebooking flights to take advantage of lower fares then I will.
Airlines make it notoriously difficult to redeem rewards.
It is inherently in airlines' vested interest to make redemptions difficult enough that many miles expire or that they can devalue our miles in the meantime.
This is post 2020 stuff. Prior to 2020, award seats were much easier to find and readily available if you searched a few months out. Now? Mass chaos. Everyone and their mother has an Amex Plat, plenty of miles or dollars to spend on premium seats. The game has changed.
Re: brokers:
The fact of the matter is that many brokers built their own scraping tools that don't have many of the rate-limiting features o seats.aero. in addition, seats has worked to identify broker activity on their end, and attempted to reach out to AC and maybe other airlines to see if they wanted to work together and share some sort of broker blacklist. This feels way more like AC just being unhappy with the number of high value redemptions that are made as a result of these tools - and I think there's something in that FB interview Lucky mentioned that alludes to this.
I don't know anything about it, but I am surprised to read that there is good precedent that scraping websites is okay, whatever the terms and conditions.
If this is true, why don't third party sites show Southwest fares?
Tell everyone you have no idea how IT infrastructure works without telling everyone how IT infrastructure works.
In the end, a court will decide. Until then and while the case might take some time to resolve, the court can issue a temporary injunction prohibiting seats.aero from including Air Canada in searches. Might want to consider Plan B if this happens.
Malfriend, I can only assume that you've never written a scraper script, nor have you ever helped manage a resource being scraped.
A competently written and lightly used scraper puts no great burden on a robust resource, true. But notice all the qualifiers I used. A black hat points broker has no incentive to do anything other than hammer the target hard and repeatedly, because if he doesn't then other black hat brokers will, and eat his lunch. It's a race to the bottom.
Tell us you've never administered a public server without telling us you've never administered a public server.
Yeah doesn't Ben have a financial interest in point.me? Why not make reference to that when talking about competitors?
And why doesn't Lucky mention points.me in the article, which at least at one point he had a financial interest in via points pros.
If the airlines made it easy to just see a whole calendar of available flights, ppl wouldnt need third party ones. But no, their sites almost always suck. They will even stop working after youve gone thru 20 days or so as it thinks youre a bot.
Agreed the advent of seats.aero probably screwed up Air Canada’s business model of selling points on soecial and then making it hard to redeem those points. There’s probably a broker problem as well that air Canada should address.
I’m not buying any more ac points. Too hard to redeem, high fees, if something goes wrong you can’t get through to their call centre for hours. Not worth the hassle.
Will not be giving them any more money in their points sales so they can capitalise their business with my money. Nb I was going to cancel my seats.aero account but will keep it on so Ian can fight this case.
Not surprising I guess, I feel like Air Canada is one of the airlines making their site harder and harder to use, rather than easier. You used to be able to see a week at a time, and apply filters for partners, etc. And all of that is gone now. Their searches turn up mostly mixed cabins and their own metal at variable rates.
They wanted to be a top rate loyalty program but then quickly dialed it back once it became popular. I hope Seats.Aero wins!
Probably their profits in the points division are sliding due to too many premium redemptions. Clearly seats.aero, and others, make it easier to find those redemptions. Most people in the points game aren't in it to redeem their points on long haul, economy flights. AC are probably getting push back from their high end partners like SQ as far too many premium seats are being bought via aeroplan.
They do. Skiplagged for example shows Southwest fares. It's just that most third party sites use fare systems that Southwest doesn't push to instead of scraping.
Seats.aero screwed up by establishing itself in Delaware. If they were outside the US it would be way harder to sue them.
Some carriers already do something very similar to this, e.g. Singapore Airlines. These are perfectly reasonable restrictions. The problem is that it's really difficult to identify people and the brokers will find a way of using fake IDs.
This is a silly suggestion; this is a small company run by ~2 people. It’s quite a bit of resources to incorporate somewhere overseas and it’s absurd to just presume they’d even need to do this for fear of a lawsuit, that isn’t something that’s really happened up to this point.
All the sites using some sort of webscraping will be at risk, not just seats.aero. So that means anyone who has a service charging people to find award availability using webscrapers may be exposed. Maybe they all got letters from AC but the others just aren't saying anything.
Will be an interesting case as what is the difference between google flights and all the OTA's going on and getting cash fare information off the Aeroplan/Star Alliance sites vs just award fares? Is it just award fares that AC don't want scraped? That would be bizarre. How would that stand up in court?