Dynamically Crop image with JS

1491 views javascript
7

I'd like to extract a conversation for a game with JS (plain) and get just the text bubbles. This isn't that hard just to crop and image, but sadly the image's dimensions and ratios is going to change. Here's the image before and after

Original (messages removed):

screen shot

And then the auto cropped:

auto cropped

How do I crop this? All the text is preserved but nothing else is in it but the text (I need this because I'm converting it to text data via tesseract and it gets confused on some other things in the image). My goal is to have this entirely client side so no PHP could be used. The image ratio and dimensions are also going to change. It's always going to be landscape but some are practically square while others are really long and short. Is this possible. Could I use some sort of AI model to capture it? Any help is greatly appreciated. Thanks!

answered question

It would really help if you'd add some code to your question...

@DragosRizescu It's hard too. I don't know which library would do this.

if you're capturing the chat why not capture the exact text from the bubbles to begin with?

@PaulGMihai I'm not sure I'm following you. If I just use the pure screenshot (the only input) to tesseract it makes a lot of errors and the conversation is practically irreplaceable from the other text. If I were to use the cropped image I does the conversion perfectly.

Wrong approach, your screen scraping by literally taking a picture of the screen, attempting to crop it and then running it through ocr software, instead of intercepting or hooking into the api response from the games server which you would have your text as-is, no AI needed.

yes but why go though all the trouble? Why not hook into the game chat api or capture from the browser the exact text via js?

1 Answer

9

Before I answer I should note a few things. Firstly, this is a pretty terrible way to do things, hopefully there is some javascript you can hook into, or maybe it even displays things to the DOM. I don't know the game, so I can't tell you, but you should definately look for alternate solutions than just scraping the visuals. Secondly: without knowing what libraries or languages or code that you're using, it's a bit tricky to provide example code, but...

The best way to do this would probably be first looking for easily identifiable corners of the screen, and using those to get the area that you're looking for. Something like that top left corner where it meets the top bar. Maybe get the bottom right from the speech bubble or just use the literal bottom right hand corner.

Alternatively you could try to figure out how it scales and edit your code appropriately, but that isn't a fun approach...

posted this

Have an answer?

JD

Please login first before posting an answer.