4

Not sure if I am right but lets assume I have an REST API and endpoint that, for example, creates some resource, lets say @POST to create User.

How to protect my app from users that do for-loop with like 10000 API calls to create useless resources?

Is it possible to write a filter that blocks such behaviour? I hope you see what I mean.

nvoigt
  • 75,013
  • 26
  • 93
  • 142
azalut
  • 4,094
  • 7
  • 33
  • 46
  • @KevinB it's a programming problem, if you consider score-based/pricing system on an app-level (or even server level) – strangeqargo May 23 '16 at 20:35
  • 1
    Looks like you want something like a [leaky bucket filter](https://en.wikipedia.org/wiki/Leaky_bucket) or something similar. We implemented something similar using the authorization credentials which we cache into a backing evicting map. If more then n requests from the same client within a given amount of time are received, we respond with an error on consecutive requests till the cache has at least space for one further request. Though to simplify thing, you can also use a fixed timer and clear the cache every n minutes automatically – Roman Vottner May 23 '16 at 21:03

2 Answers2

1

You have many options:

  • block ip when certain ip generates a lot of queries
  • block ip+useragent+other details about user's browser
  • block user using behaviour pattern matching
  • obvious choice is to hide some REST methods behind an email-verification (registration process)

Let's say in pseudocode ABUSING_ID = md5(ip+useragent)

  • you can have something like pricing system, so every api call has a price (0 to 10), and store every ABUSING_ID+their api call price in a database (i.e. for a day). So, when total price of requests for given ABUSING_ID goes over the threshold value, you block them. (More formal, and, I'm sure, better approach is what @roman-vottner proposed Leaky bucket algorithm)

You can implement this schemes on different system levels (client-side, load-balancer/web-server side(fail2ban), application-level)

More levels you cover the better.

  • some beginners can use gui-automation to click inside your UI (if it exposes your API), so you need client-side protection.

  • more advanced frauders can use scripts to curl your api, you need server-side protection

  • another kind use something like phantonjs to emulate browser environment (client-side protection)

  • some tycoons know enough to hire cheap workers to click on your site/make requests with custom-crafted tools

  • proxy-servers/tor/botnets/browser-fingerprinting prevention (basically, changing user-agent and other details with every request) etc.

If your API is in heavy usage, you can start with statistics gathering - it will pay back later. In the worst case you'll need a team of data scientists and coders to create a working, hard to break fraud-prevention system.

It's whole world at war :-)

P.S. I didn't say anything about authorized API calls (tokens etc.) because authorized API calls are much easier to catch, we're talking about unprotected REST queries here

P.P.S. There is another metric, but you must think of it as a last resort: if your service is international, global-level, but you have too many fraud from, say, fictional Fraudistan, and this fraud harms your business more then good traffic gives profit, you should block entire country and make a notice, so good customers from that country can register through a stricter protocol.

strangeqargo
  • 1,276
  • 1
  • 15
  • 23
0

I agree with @strangeqargo's answer. If you don't want to implement this yourself, there are several services which will analyse requests and recommend whether or not to block them, I've had a pretty good experience with ShieldSquare

Axel
  • 49
  • 5