holy shit i just pulled off something incredible. i didn't know if it was going to be doable but i JUST BARELY got there haha i feel like a genius

the fucking adrenaline rush from making this work lmao i feel so good

long story about how i cheated at a recreational programming challenge 

So I've been trying to solve this Codewars kata. It's a recreational programming challenge. And the challenge is to write a function calc that takes in a string argument and evaluates it as a mathematical expression, following BEDMAS etc. Not too hard to solve properly, probably. Maybe a little tedious, you gotta build an expression tree or something. Whatever.

But the thing is, there's a function in python that already does that, called eval. eval is usually bad practice for a number of reasons but in this case, recreationally, iId love for my solution to just be calc=eval and submit it that way.

Of course it's not that easy. The code you give as an answer runs in an environment that the challenge creator controls. There's a regular expression that tests for the use of eval in your code and if it's there, your code doesn't even get to run.

I thought "that's okay, eval is a global function, and you can access anything in the global scope via the globals method. I can pull the function out of there and call it without ever using its name, so no source-code based checks can stop me".

Well, no dice, the author also thought to ban globals.

So I decided to take a different approach. There's a python module called 'builtins' that contains a copy of all builtin functions, for if they get shadowed or something. If I import that, I can grab the eval method out of the module object and solve the challenge, right..? Well, you may have guessed that builtins is also on that list of banned words. But I wasn't going to give up on this method without a fight.

I've been using python since I was 14. A lot of that has been code golf, another kind of recreational programming that often requires extremely esoteric knowledge of your chosen language to do well in. I am very familiar with python's quirks. For example, I know that you can use import modulename to import it into the global scope, OR you can directly do what the syntactic sugar is hiding from you by calling __import__('builtins'), which returns the module object the same way any other function returns values. Sure, I still have to write the word "builtins", but now I'm dealing with strings instead of python names. I could get around this check with __import__('buil'+'tins').

This is actually true. That check was no longer stopping me. But it turned out that there was a second method to prevent cheating, a much more sophisticated one that hijacked the way python imports modules. It didn't matter how I did it, it didn't matter what syntax I used, the underlying function getting called in each case had been extended to outran ban specific modules, swallowing errors and covering its tracks to prevent you from learning anything about how the anticheat works.

This was very nearly the end of the line for me. In fact, I've gotten here before, confident that there was more to be done but no idea what it could be. There's no way some random challenge author covered every possible hole, but they probably covered more than some random asshole (me) is going to find...

...but I still had one more lead. If you tried to import a banned module, it didn't just fail, it threw an exception. And in the python trackback, I could see the filename of the code standing in my way. I would love to pop open the anticheat and actually read it to find a way through. There was still a problem: open was on the banned keywords list.

My next bit of progress was more luck than skill, to be honest. I was actually barking up the wrong tree entirely. I thought that maybe, there might a module somewhere in the standard library with an obscure way to open files. I was just browsing the available modules looking for anything that caught my eye, and I stumbled into the tokenize module. It does have the ability to open files from disk, and it's even supposed to be used to open python source files, so surely I could get the code out of it somehow. The problem was that the method was called open, so the same regex test would prevent me from using it, too.

It wasn't until I was browsing the source code for that library that I realized: It has a method named open. It's shadowing the name of a builtin. So if it wants to open a file, it must have made a backup of the open function under a different name. Sure enough, tokenize._builtin_open is a backup of the open builtin, and I could use it to view the anticheat code.

The rest was pretty trivial, but even my instance's absurd character limit is about to run out, so I'll have to put the remaining part in a reply.


long story about how i cheated at a recreational programming challenge (conclusion) 

It turns out that the tests that actually call my function are also run in the same environment as my tests are. I mean, they have to be, that's how they have access to my function. But the author of the challenge wanted to test submissions against the simplest possible answer, to prevent bugs. So the tests themselves are implemented with eval. That means that there still is a backup of eval present, but the way it's hidden is pretty devious. I'd never have gotten it without source access.

Remember when I said that __import__ was overridden, which fundamentally overrides how python handles imports at all? That wasn't just to ban some modules. They used it to smuggle things they needed into the test environment without giving access to the user. If you called __import__ with a specific string, instead of importing anything, it returned a pair of saved values. One of those values was the jackpot, the single function that trivializes the challenge.

My final submission:

def calc(expression):
# Thanks for the challenge, it was a blast
return __import__('extract.randomPassWord5752FFG2')[1](expression)

Surely, SURELY, you can see how much more fun this was than writing a function to solve math problems.

Β· Β· Web Β· 4 Β· 8 Β· 32

it's like playing a CTF except there wasn't supposed to be a solution

@monorail Congrats, you solved the hidden question. You've unlocked the secret exit.

@prplecake they said "don't do it again unless you want your account banned" which means they're not going to ban me today :P

@monorail i would've simply given you the job to improve the anticheat :P

@monorail @prplecake "It seems like maybe your anger is directed at the wrong person. I've provided you with a learning opportunity and shown you how to improve your code. Surely now Codewars will be more secure -- unless, of course, you direct your energy toward being angry at the person who showed you how you'd erred instead of toward fixing the problem."

@noelle @prplecake nah it wasn't a security thing, it's all sandboxed or whatever. just sportsmanship

@monorail @prplecake Oh, I know, I use Codewars myself. I just like being hypothetically smug at people who get mad at hackers who find flaws in their carefully-constructed gardens. ;)

@noelle @prplecake ah gotcha

they were like "this solution is invalidated" and i didn't say this but i was thinking "that's fine, you didn't invalidate the fun i had"

@monorail holly, I mean this in the absolutely nicest way possible: you're an asshole :)

long story about how i cheated at a recreational programming challenge (conclusion) 

@monorail this is amazing and I love the way you wrote it

@monorail wow, this was a very exciting read and for once, I think I actually want to learn how to code. This is really, really cool!

Sign in to participate in the conversation

A general fediverse instance for people who generally like pokemon at least a little bit. Newly registered users must be manually approved due to an increasing number of spam bots; if you look like a person, your account will be approved as soon as possible.