The AI-corn Project - How good are these LLMs anyway?

 

Over the last few months LLM's have reached a level where many developers (and almost everyone else) believes that these tools will (or have already) change the very nature of our work as developers. Specifically LLMs are increasingly able to do implementation work - that is the actual 'feet on the ground' coding for many tasks - so long as they are well guided architecturally and structurally. 

Intent

The purpose of these blog posts is to establish (for myself) whether I believe the current LLM technology is capable of generating code at this level - specifically if it can write algorithm and source code to do some novel but well understood and well-defined tasks with the same guidance I would give a very junior developer with no real world experience.

Additionally I'd like to learn how best to use these tools personally; what they are strong at; what they are weak at and how I feel about the experience of using them this way.

The Task

This year I entered the JS13k games contest as I have done for many years - My final entry can be found here and I was really happy with it (sorry about the colour scheme - the theme was 'unicorns and rainbows').

JS13kgames has some tough rules: your final source code must be under 13kb; no external loading of assets or code of any kind; must run mobile and desktop browsers; 30 days from start to final product.

This year I spent about 50 hours on the project over those 30 days and had spent some time before the contest started doing some research. Much of this time was is game design and software architecture - perhaps less than half spent on source development. I have demos from many different days in the project so I have a record (and git) of how fast it went and how it looked as the time progressed.

The game is nothing terribly new conceptually like most other platformers; the physics is a bit unusual but easy to explain; the graphics display is all constructed from text using the DOM; the level definitions are simple text strings; other than the player avatar there are only three other 'sprites' which are very simple.

So the big question is would a state of the art LLM have made my life easier if I got it to do the implementation? 

Is a LLM capable of writing this code? Can it put together the code into a complex working game with levels and high scores? Can it write basic interpolation, animation, timing and utility services given that it is not allowed to use external libraries? Is it capable of handling terser and zip and managing the code size limit and how much of the burden around this tooling can it remove? How much debugging will I be left with?

The Plan

I've spent a few days researching the new tools (and am familiar with many of the tools) and have decided that cursor back by some token allocation seems to be the base setup most folk use for serious project work - I have that all set up.

My plan is to follow the same steps I took when doing the game jam. The design work is done; I know at least one way of solving every technical problem; I know what I want it to look like and how I want it to work. So it should go really quickly.

 

Starting with a few small (tiny html+code+css) proof of concept projects just to show I can make the technology work and my ideas for rendering the game are practical. I plan to do this phase in a fairly minimal project in cursor and learn to find my way around there first.

Then I will 'start' the contest - set up a serious project that will develop towards my complete game; build design and specification documents; rules sets for the unicorn; feature by feature requirements breakdowns. 

From here I'll try to see if I can keep up with my last attempt - try to get similar demos and see how it goes. Ideally I would never have to touch the code directly but I will do some debugging or coding before I give up.

Wake the Cat - Day 27 - Nearly There

 Got some more sound effects; got some more levels; a pretty new level display and 11 out of 13 levels complete!

BUT am currently sitting at 12.7kb / 13kb - very close to the limits! I've got to create two more levels and have a few details to fill in but here is a demo 3 days before the deadline.

Wake the Cat - Day 24 - Level selection and some levels

 Finally getting to a form of the game that has some substance and overcome most of my most serious technical hurdles:

a) WebXR interface: Made all this work although I had a little trouble with the emulators. You can enter the game as an XR session with appropriate hardware - I have look to select so that if you look at an object for a bit it gets clicked with a sweet little torus as UI. I also implemented some rotation controls for the puzzle in the VR world (look down a bit to find them) - they could use some work aesthetically but they're functional. Also you can fast select using triggers or buttons (if your hardware has buttons) and rotate the world using the joysticks (if your hardware has joysticks). I've played through the game with my 'cardboard' headset so I know it can be played with the oldest hardware!

b) Level selection and switching between levels.

So it's beginning to look like a game - here is a demo with 5 levels implemented. I'm not sure these levels will be my final versions we'll see how we go.

Wake the Cat - Day 21- Almost a Game

 Lots of work going on on the game:

• A level selection world and switching between levels and returning to the level selection pretty elegantly.
• 5 whole levels implemented - some of them pretty good
• Some basic work on the VR hardware integration - just setup not implemented yet but the render loop is XR-API ready and working and I got some pretty buttons.
• Win conditions all set up and working (just not exciting)
• Built the simplest tenser.js and zip pipeline for module level compression which is going to be good enough I think.

So here is the demo of the first playable version....size is currently 7,486 bytes - so that is looking okay.

Wake The Cat - Sound FX

 Did a bit of late night vibe coding for sound effects - interesting and successful. I had to refactor the chatGPT code to make i) better and ii) more js13k compatible but most of the basic ideas and sound engineering stuff (filters / distortion etc) was new to me - so it helped me make good progress in very little time.

Here is a little demo of some sound effects for the game.

Wake the cat - New Demo - with a cat - Day 16.

 Been a busy weekend - lots of new work.

On the windchimes side I'm really getting somewhere - this demo has easy working access to all kinds of windchimes tuned by culture and soundscape - I think it's pretty cool and the general topic of culture and musical keys is really interesting. I'm happy for this as a background soundscape solution for different levels.

On the game side I made a couple of extra pieces: a rotational ring thing that goes round and round; a static cat platform; the ability to draw text in the world on signs ; and I finally made a cat!

I'm not really happy with the cat shape (bull-terrier-shape?) nor the implementation and I hope I'll have time to do it again. However I am pleased with the generated fur textures, the eyes  and ears, and the ability to make different cat-coats (although the tabby pattern is a bit weird still).

The system can also generate different levels from a level factory and has a 'wake the cat' success outcome (which just logs to console so far). So I'm getting there, but it's day 16 so I'd better be!

So here is a demo of the latest 'test level' with a whole whack of different objects. 

The big outstanding technical hurdles are: VR hardware integration (I've got a pretty good idea how to do this but there is time, technicality and work here); design for another 10+ levels (long, hard, detailed work); sound FX for moving bits and the cat (no idea; but hope to mostly vibe code it with the Web Audio API);  and setting up a tool chain for compression (I've done it before how hard can it be?).

So far so good but my other life is looking busy!

    

Wake the Cat - Demo 2 - It Moves! Day 15

 Got the movement and some mechanical pieces for my new game. 

This demo shows my piston like piece (three state expansion), a simple moving needle and an static cat basket.

The pistons demo how pieces can be attached to each other; and the needles are set up to test the detection of collisions between objects (which causes them to 'bounce back'), finally the system of passing through clicks allows the basket to activate the needle beneath it. All important bits for my game.

Still no cat though!

Wake The Cat - First Demo

 Have been getting some work done on basic elements for my puzzle game - here is a demo showing a cat basket and some pointy knitting needles - all textures generated using canvases at run time and the bump textures looking pretty good.

So no actual games yet but a working camera, lighting and world.

Windchimes for Wake The Cat

 All my previous attempts at js13k have been pretty weak in the audio department - I rarely play games with sound on and it's always a problem with the 13kb limit (no samples so everything sounds a bit chip-tuney).

This year in order to break this streak I thought I spend some time on it early. My game 'O' had a beautiful, haunting soundtrack (not made by me) which might best be described as ambient, dreamy piano - just the sort of thing I love.

For my new abstract game I was hoping for something in the same spirit; but generated by algorithm. I had in mind the quiet ambient sounds that a large, well tuned windchime might make (if you hate windchimes you've probably only heard the small, tinkling badly tuned type). With that in mind I set up a little demo of a set of Pentatonic chimes here. I'm kinda liking it and think it might take me a long way towards background sound. You can play with it here if you like.    

JS13kGames 2025

 The time has rolled around for a new js13k games - it started while I wasn't watching a couple of days ago but I had planned to enter this year.

The theme is Black Cat which appeals to me - I've entered almost every category over the years but never their 3D webVR section so I thought that is what I'll do this year. 

It has been more than a decade since I used Three.js which it a tribute to flexibility and maintainability of that project.

But the last time I did use it I made one of my favourite games ever called 'O' - this non-verbal non-text puzzle games was a really cute and efficient piece of programming and I'd like to set up a similair abstract puzzle games for js13k this year - I think I'll call it 'Wake the Cat'.

js13k2022 - The Devil's 4-Play - Completed!

 I have run out of my very short time scale for this years js13kgames contest. My final product is called "The Devil's Four-Play" and I am quite happy with it. This new version has some nice flavour text and pretty icons with a 'hellish' theme.

You can play it online using this link.

I am very happy with the outcomes of the project - it has all the features I imagined and looks polished. 

I only used 8kb of the 13kb available and did the project in about 20 hours of development.

This was an intense 4 day project for me because of my travel plans.

Good luck to all the other competitors and use your remaining days well! 

js13k2022 - The Devil's 4-Play - Almost there!

 Had a few more hours after work yesterday to put some pretty graphics and theming on the game and added some new sounds effects.

I've decided to called it 'The Devil's 4-play' and themed it much more hell-like.

Here is the latest version and the long term archive version here.

I still need some extra copy on the intro and help sections; and I'd like to make the sounds better - but I have to finish in the next 24 hours so we'll see how it goes.

JS13k - day 2 - almost done

 In my super short time scale js13k this year I've spent much of the time I've got available but with some  solid hard work today I've got a good ai playing at various levels of skill.

Have enhanced some of the graphics and made it a bit friendlier. My current live version at https://js13k-fourplay.herokuapp.com/ . Archived version on github (no online play).

Tomorrow I want to display moves and scoring more clearly, take a look at how to theme and colour the game and work with the audio a bit.

JS13k 2022 - First Version

 Been hard at work in these first hours - here is a first working version of my tic-tac-toe like strategy game.

I started with my own turn based game starter project , took a lot of inspiration from this codepen and did a bit of work with css perspective to get a working (mostly responsive) board layout.

The AI player doesn't work yet but you can play another human online or locally and the rules are explained (somewhat) and implemented. The latest working version is served for multiplayer here. Local  play only archive of this code is here. 

A long days work - tomorrow I program the AI player.

JS13k 2022 - Starts Now

 For several years I've enjoyed the js13kgames contest and have consistently created an entry.

This year is a bit difficult as I am adventure travelling for 27 out the 31 days of the competition thus I intent to submit my final product 4 days from now and find out about the results when I return in 5 weeks.

This year I have 10-15 hours of development time over 4 days as opposed to my usual 30 - 50 hours over 4 weeks. So that is a constraint.

This year I'd am planning a simple turn based game with these features:

• Play 2 human players on one device; play two human players online; play vs AI
• Have varied and worthy AI opponents
• Abstract strategy game
• Some simple audio effects
• Glowing, tron-like, neon 80's appearance

The contest is started - the theme is 'Death'. 

JS13k Warm Up 2022

 This years js13k will be a bit different for me - I am travelling for most of the 30 days and it seems unlikely I'll spend more than 15 hours on the project before I leave. However I've completed every year for 7 years and would like to do a simple respectable entry.

Last year I spend a significant portion of the time setting up infrastructure for my project which was required to play in three modes (player vs player local; player vs player online and player vs AI) and the UI interface to support these modes including a lobby where on line players can meet.

In order to avoid this overhead this year I decided to do a public prep framework with some basics required to support a two player turn based game. This new structure makes extensive use of async/await to make the game code very readable; it also support basic audio support; a menu system;js13k style sockets support for multi-player and some basic DOM manipulation code.

The 'game' in this example is a trivial game where each player takes a turn and the game ends after 5 turns.

A running installation can be found here  and the code is publicly available here.

The Last Days of JS13kGames 2021

 One of the things I didn't mention in my previous blog posts is that I intended to take 8 days of the contest period off from technology; thus I spent days 19 to 27 in the mountains without connectivity generally relaxing, having fun and doing nothing too intellectual. This was part of the plan but I was a few days behind on the plan when I left. Most especially I had not completed the AI work on the game which was supposed to be done some days before - the AI was pretty dumb and it was hard to understand the game well by playing against it.

I did write the tutorial text and some design but probably only spent 3 hours on it in the week away.

When I returned from the wilderness with just three days to go the game looked something like this...

At this point I've got a long outstanding feature list and am already looking pressured. But first the AI needs to work - it took a bit more effort than I expected (and a lot of time to tune the heuristics) but I got to a place by the next day where it satisfied my goals:

• That it could beat an experienced player in it's most analytical modes.
• That in less clever modes it would play in a human-like manner - make mistakes; play towards positions that aren't that great etc.

Here is an example of it two versions of the AI playing against each other.

With just a day or so to go I needed to tidy up the interface and test it thoroughly both as a game and as a piece of software that had to run on multiple browsers, multiple os's and multiple device forms. This is when I had my worst moments on the project...

The responsive aspects had some problems (especially around more square devices like an iPad) but those I have extensive experience with and understand (media queries for the win) and had expected.

It looked terrible on Safari (flickering z-order/backface issues) but it worked and I had expected issues on Apple software (aren't there always). 

On Firefox the perspective view simply didn't work - after some investigation I found that google and Mozilla's interpretation of the transform-style property and how it is inherited are somewhat different. With a sigh of relieve I explicitly implement the transform style at each level of the DOM and then I get perspective....but it totally failed in multiple ways - the graphics wasn't appearing clicks weren't working and the lobby systems were unreliable. I thought I had played nice with the browser specs and had no expectation of having problems with a highly standards-aligned browser but could see  I had some serious problems that were going to need both research and restructuring of the graphics systems.  

It took me 5 hours of hard slog that I didn't really  have - several different aspects of the implementations of 3D transforms on svg and DOM elements are understood differently by these browsers. Firefox (probably the more standard compliant browser) really doesn't like CSS-level transforms on svg path elements -  mostly around ambiguity about the transform origin for such an element. I had to introduce several more divs for 'rigging' to manipulate the svg leaf-shapes efficiently in a cross-browser way.

Then I resolved most of the Safari issues; for me this is a slow and inefficient process as I only own oldish apple hardware for testing and don't use it on a day to day basis. But I made it work on my mac-mini and my test iPhone 6+.

After that I fixed all the little things (message if the mobile screen is portrait; messaging around players bailing out of multi-player; messages for empty lobby  and about 10 other small features).

Then just setting up standard board sizes and starting positions. I decided on 4 boards for the game - the boards are different each time but the size and initial elements on the board (but not their position) defines the courtyard. I played around for a few hours with peice collectiosn until I found something I mostly liked for each board.

Finally I changed the colour schemes a lot as I had always planned and finally had a release version.

You can play that from the js13k site here. 

I submitted this game to all three categories I wished to (desktop, mobile and server) a few hours before the close of the contest  - in time is good enough but not how I like to play.

I've taken a brief look at some of the other games and the standard seems exceptional this year - between the skills that various players have developed and the new emerging browser technology it gets more difficult and competitive every year. As always there are several competitor who have shown amazing design and technical skills that I am in awe of. 

This was my 8th year I have competed - it's been a hard year for me but I'm proud of the outcomes.

JS13k - lobby and front facing Code - Day 17

 I've been working day by day adding small features to my game. It has a lobby with a pretty menu system and the first audio effects and the look of the game is slowly improving.

 Currently all modes work (p1 vs p2 local or offline and p1 vs computer) on the heroku server - this version will remain live in the long term (but has no multi-player online)

A quick screen shot of the lobby area:

This is what the game is looking like:

JS13k - A Weekend of Appearance - Day 15

 Halfway through the contest time and I had a weekend day to do nothing except JS13k so I thought I would pay some attention to the appearance of things - I knew the graphics would be very important to make the game come alive for people so I did an intense 14 hours work on the graphics of the game.

This is what I started with at 1pm on Saturday:

This version which you can play here is from about 6pm with proper cracks, animated piece movement and stems:

by about 9pm I had the growth  animations going and pretty randomly distributed leaves 

much later that night I had tiles drawn properly and better cracks:

Still not totally happy with the graphics but I think it looks better and the hard code issues are cracked.

Current compressed version is about 48% of the 13k.

JS13k - Almost an AI - Day 13

 Today I have my first attempt at a dumbish AI player which has turned out very well - it's simple heuristic is getting dominance of the board in the very short term as defined by the score.

Here is a version where you can play 7 x 7 against it - a board where something like this AI sometimes does okay.

So I'm getting close to my goals for a game that can be played between players together or apart and would support single player vs AI opponents but I need a menu system and intro page to structure all these pieces. 

Currently the project is running at 4880 bytes compressed - about 37 percent of the allowed package size. That's okay I think - there will be space enough to make it pretty.