I'm going to make an ML bot want to help?

[woodyet]

Hi all,

I am currently doing a PhD in AI, focusing on interpretability, (basically understanding why ML algos make the decisions they make).

I have seen that there has been some success with an alpha-go inspired TD bot. And whilst very cool, this approach this will not work for most puzzles in the game, alpha-go's learning came from playing against it's self many millions of times to get better... and most puzzles are one player so an iterative traditional ML approach will be adopted.

For my first attempt at this I need a puzzle that is easy to simulate (so I don't actually have to play the game to train the model).

And I need a puzzle for which the scoring is very well understood (so the model knows which moves to prioritise).

<strike>I was thinking bilge to start?</strike>

Nah lets do carp... because it's harder xD

Please let me know if you can think of any better suited puzzles.

The code will all be written in python using tensor-flow... Why? well this is what I use for my research and if you really want to package it up later on for easy consumption there are tensor-flow library's for C++/Java that work with linux and windows.

All of this project will be open source and I will update below with a git hub repo once the first commit is up.

https://github.com/Woodyet/W.O.O.D

Note: Please do not come here to ridicule my coding, it is not my job to code; I think of a problem and then get the program to fix said problem, everything in-between is an inconvenience.

Any help with links to how puzzle scoring is calculated would be much appreciated/ If anyone has deconstructed the YPP client and can give me definite numbers... well now that would just be perfect.

------------------WORK TO BE DONE---------------------
-Gather many many carpentry boards (these will be use for training) over 50,000 would be nice
-Found the code to generate the boards on the fly... so lets use that instead.
-Make basic simulation of carpentry puzzle ----> This might help https://madolinn.github.io/pp/index.html
-Work out what heuristic should be used to train the AI

Train.
Test.

[Mehfailz x3]

Do Carpentry and let it solve the holes, prioritizing 2 adjacent holes at a time with a turn counter (so it knows when other holes shake).
Make it happen and become rich.

Edit: If you are looking for simple and easy puzzles do Alch or Distilling.

[woodyet]

Sounds good, I suppose bilge has been done to death. This is now a carp AI .

Now to figure out how carp is scored....

[Dan]

Type Count Prob :I
---------------------------
P 704 0.3793 22.0
F 224 0.1207 7.0
Y 224 0.1207 7.0
L 128 0.0690 4.0
N 128 0.0690 4.0
T 112 0.0603 3.5
U 64 0.0345 2.0
V 64 0.0345 2.0
W 64 0.0345 2.0
Z 64 0.0345 2.0
X 48 0.0257 1.5
I 32 0.0172 1.0
---------------------------
Total 1856 1.0000 -----

P = 22
F, Y = 14
N, L = 8
T = 7
U, V, W, Z = 4
X = 3
I = 2
Bucket O' putty = 1

Based on these probabilities, the way pieces seem to be generated, is, there is a central piece (the piece with the black dot on it) and, there are 4 open nodes. Then, at random one of the 4 open nodes is picked. There is then 6 open nodes, for all of the un-connected edges of the shape being generated, and it randomly adds to one of these 6 and so on.

edit: Carp is pretty solvable without ML. Look into Dancing Links by Knuth.

[woodyet]

Thanks for the numbers! They will really help.

They are essential actually, as I am currently using AI gym (made by google). To make a fake carpentry environment that the AI can train in.

I am aware that you can probably do this very well without machine learning, I am doing it as a test of my ability. And hopefully the skills I learn can transfer to other puzzles.

Additionally because i'm using ML I might leave the learning of the algorithm on, so that when you start the bot it uses a general model, and slowly gets better as you let it play (with some kind of hosted location for all of the models).

Additionally his might help with bot evasion, as every client will play slightly different.

Dan, do you have any ideas what the parameters of the 4 holes creation is? I can just get this from the game (opening and closing the puzzle and recording the shapes of the holes), but i'd rather not.

edit: I have figured out how the holes are generated

[woodyet]

Update....

I have decided not to make a virtual environment for the bot and instead will just teach it with the pixels from the game.

I will limit it's ability to click to inside the puzzle.

I think the best way to get feedback from the puzzle carpentry is to use the sounds cues from the puzzle... they can be found here

AppData\Roaming\Three Rings Design\Puzzle Pirates\rsrc\bundles\media\yohoho-puzzle-duty-carpentry\media\yohoho\puzzle\duty\carpentry

This is quite a nice way to rate how well the bot is playing as pigs breakfast to masterpiece all have their own sound bytes.

The general case for the flow of the program will be...

>>Get Screenshot of play area
>>>>Pick Piece To Place
>>>>Choose Rotation
>>>>>>Place Piece
>>>>>>Listen for sound cues
>>>>>>Calculate Reward For Move.
>>UPDATE NETWORK
REPEAT.

With this change the bot has become a lot easier to code... and has setup quite a nice platform for this system to work for any puzzle that has its sounds directly linked to the score you are receiving (I think this model will transfer well to Bilge/Dnav/Gunning(With some tweaking to figure out when to use the wash)).

Hopefully I will have the prototype working by the end of the week.

Additionally... I will test how good this bot is first before releasing the source, I don't want to release this if it breaks the game (I like the game) but I will hopefully have a video up soon .

-------------------------------------
CARP GAME STRUCTURE
-------------------------------------
For 800x600 screen.

If a single block of a carp piece is considered one square the dimensions for important areas are as follows

Board size = 23 x 31
pickup area = 15 x 5 ( or keys 1,2,3 )

Network inputs

4 Float values indicating if any of the holes are complaining
after 5 turns with no peiece placed turn on @ 1/3
after 6 turns with no peiece placed turn on @ 2/3
after 7 turns with no peiece placed turn on @ 3/3
reset @ 8
1 x screenshot

Network outputs
pickup area = 1,2,3 (argmax)
placement area = 23*24 = 552 (argmax)
rotations = 0,1,2,3 = (argmax)
flip = 0,1 = (argmax)


Score sound cues

hole_masterpiece = 100
hole_craftsmanship = 70
piece_place_perfect = 70
putty_use = 65
grain_bonus = 50
hole_fair_job = 40
hole_sloppy_work = 0
piece_place_overlap = -10
hole_pigs_breakfast = -20
hole_grows = -75
piece_fly_off = -75
NOSOUND = -200 (indicates invalid placement)


PS ANYONE KNOW OF A WAY TO EXTRACT JUST THE CARPENTRY PUZZLE FROM THE CLIENT TO DO TESTING ON????

[CrimsonVex]

PS ANYONE KNOW OF A WAY TO EXTRACT JUST THE CARPENTRY PUZZLE FROM THE CLIENT TO DO TESTING ON????

I think the best way to simplify carp for a ML algorithm would be to read the carpentry puzzle using a basic colour bot, that turns each hole into a basic input of text as an array.

See my crude demonstration below:

[Devereux]

Update....

I have decided not to make a virtual environment for the bot and instead will just teach it with the pixels from the game.

Please don't do that...

[woodyet]

I think the best way to simplify carp for a ML algorithm would be to read the carpentry puzzle using a basic colour bot, that turns each hole into a basic input of text as an array.

See my crude demonstration below:

[ATTACH=CONFIG]2293[/ATTACH]

I will do image processing so the bot will basically sees what you are describing, I'm setting this out in the way I have described so the bot can easily transfer between puzzles (Hopefully this same approach can be used on many puzzles), whereas if i focus on carp I only solve that one puzzle.

Please don't do that...

Why?


Basic edge processing I will feed to the NN

[woodyet]

Update:

I tried learning from the puzzle... It was too slow but now we know! just a lot of wasted time on my part setting up sound recognition software and stuff but I will go back to making an environment to learn in... Which will take forever.

Coding the whole of the Carpentry mini game its not going to be fun so if anyone wants to help this is the environment I am trying to achieve.

1.Var Puzzleboard

#This will be a 23 x 31 array that represents the whole puzzle screen this array will be 1 if there is not a hole 0 if there is a hole and 0.5 for each square on a piece that can be picked up (the detection for this is done already and will be how the network sees the puzzle)

2.each hole needs a generation algorithm that is identical to the game.

3.each piece in the middle has to be generated same as the game (Not hard see above)

4.Score will just be misused depending how many pieces overlap

[pas]

Off topic but semi related, does anyone know the probabilities of breakers appearing in sword fighting?

[woodyet]

Off topic but semi related, does anyone know the probabilities of breakers appearing in sword fighting?

Sorry No idea, but i'd imagine (as you get the same as your opponent) they are generated server side.. meaning searching the client won't help...

Best bet is to setup some image detection for the puzzle to see when you have received breakers then just write some code to continually play yourself and record out when you see the breakers in a txt doc? That or do it by hand :L

UPDATE

I have now reverse engineered the hole making algo from a de-compiled client, I will now implement the generation of pieces /scoring and we should be cooking with gas

I have no completed the piece generation, have to finalise the scoring and then I'll be training.

[woodyet]

edit: Carp is pretty solvable without ML. Look into Dancing Links by Knuth.

I have now decided to scrap the AI portion of the bot, it works well enough without expect a release video next week

[Money]

nothing to add except: damn, y'alls guys is smarts

[jak8222]

I have now decided to scrap the AI portion of the bot, it works well enough without expect a release video next week

Scrapped the AI portion? What will you be releasing instead?