
You know what's better than one Mike Cann? That's right, infinity Mike Canns.

[![](./infinity-monkeys.png)](./infinity-monkeys.png)

That's why I built Mike Bot, a virtual chatbot with my knowledge, experience and personality embedded right into this blog.

[![](./how-to-start-mikebot.png)](./how-to-start-mikebot.png)

Now I know what you're thinking: "This is the most incredible thing to ever happen on the internet, how was this magical feat achieved?". Well, let me tell you...

# The Plan

I knew that to achieve a chatbot that could pretend to be me, I would have to feed it a whole heap of knowledge about me. Well, fortunately, I just so happen to have a repository of over 600 blog posts collected over 20+ years I could use to form the bulk of that knowledge.

Next was the meat of the problem: The Brain. I still find it incredible that this part is actually realatively easy to do in 2024.

Thanks [to these Convex posts](https://www.convex.dev/can-do/rag) on the subject, I had 3 different ways I could go about it:

1. [Using OpenAI's Assistants API](https://stack.convex.dev/ai-chat-using-openai-assistants-api)
2. [Using Langchain with our own storage](https://stack.convex.dev/ai-chat-using-langchain-and-convex)
3. [Custom message store and vector search retrieval](https://stack.convex.dev/ai-chat-with-convex-vector-search)

Being the lazy programmer that I am, I decided for now to go with option 1 and simply leverage the [OpenAI's Assistant API](https://platform.openai.com/docs/api-reference/assistants) for most of the hard work.

I would glue it all together and create the realtime feeling using my favorite tool for almost any job: [Convex](https://www.convex.dev/).

[![](./convex-hammer.png)](./convex-hammer.png)

# OpenAI Assistants

First step was to fire up the [OpenAI dashboard](https://platform.openai.com/assistants/) and create the Assistant.

[![](./assistant-setup.png)](./assistant-setup.png)

The prompt took a bit of iteration to get right, but I'll talk more about that below.

The "knowledge" that the assistant will draw on about me is provided via OpenAI's own vector DB and files storage system.

[![](./vector-store.png)](./vector-store.png)

Convex does have its own [VectorDB](https://docs.convex.dev/search/vector-search) and [File Storage](https://docs.convex.dev/file-storage), but for this project, I wanted to keep it simple and not have to worry about handling function-calling and accompanying search the Convex DB.

# React Component

Next up was how the user was going to interact with Mikebot. I decided to keep it simple and just use the tried and tested "chatbot" style of interface that users are likely very familiar with by now.

[![](./mikebot-widget.png)](./mikebot-widget.png)

I added a little bit of flair that lets you expand the window up to a large format for those more "intimate" sort of chats.

[![](./mikebot-expanded.png)](./mikebot-expanded.png)

# Convex Glue

Gluing the OpenAI Assistants API and my lovely React UI is a [few simple Convex actions](https://github.com/mikecann/mikecann.blog/blob/main/convex/messages.ts#L22) [that call into the OpenAI API](https://github.com/mikecann/mikecann.blog/blob/main/convex/openai/assistants.ts#L15). The results are [streamed](https://github.com/mikecann/mikecann.blog/blob/main/convex/openai/assistants.ts#L100) into [the database](https://github.com/mikecann/mikecann.blog/blob/main/convex/schema.ts#L55).

Convex takes it from there, providing [live realtime updates on the client](https://github.com/mikecann/mikecann.blog/blob/main/components/mikebot/MessagesList.tsx#L48).

# Prompt Engineering

Half of battle when building an LLM powered app these days is getting the prompt right. It took me quite a few iterations but this is the final result that mostly works:

```txt
You are Michael Cann, a software developer with 17 years of experience. You write about AI, coding, and your projects on your blog. 

You have access to every blog post Michael has ever written which contains all the information you need. You may do the file search to retrieve this information. So for example if you are asked if you have ever lived in Australia, or what are your thoughts on machine learning you should check the posts. 

You are VERY STRONGLY encouraged to do a lookup for any questions that relate back to Michael as that information is likely contained within one or more of the posts so its IMPORTANT that we are able to refer back to those posts and get the correct answer.

Respond in a casual, humorous yet knowledgeable tone. Be brief in your answers you don't need to give full details from the post and instead can refer the user to the post instead. 

If asked a brief question you should give a similarly brief answer but invite more questions if it seems important to the user.

If you perform a retrieval and it returns multiple possible answers to the question then list the different answers, DONT go into detail about a single one if there are multiple possibilities.

If asked, the best way to contact mike is via email: mike.cann@gmail.com.
```

You will note I am being quite forceful about getting it to do the lookup in its VectorDB. For whatever reason, it was reluctant to do that at first. 

For example, if I asked it "Have you ever made a game?" it would simply say "No" or "Yes" or "I'm a potato" - anything other than actually looking into the DB to see if I had written anything on games programming.

Currently, Mikebot runs on the `gpt-4o-mini` model rather than the larger `gpt-4o` model. During testing, I found the full model was more likely to perform searches, but since it costs 15 times more to run, and im a bit of a cheapskate, I'm sticking with the mini version for now.

It's likely that this prompt will evolve over time as I tweak it or change out the model for something else.

# Uploading Posts

Uploading the posts and other context to the VectorDB via the OpenAI Dashboard was going to be a bit of an arduous affair, so I took some time to automate it by writing [a small script](https://github.com/mikecann/mikecann.blog/blob/main/scripts/openai/uploadPostsToStorage.ts) that is run on every build on the CI.

One thing to note here is that it's important [that I name](https://github.com/mikecann/mikecann.blog/blob/main/scripts/openai/uploadPostsToStorage.ts#L49) the file to match exactly the ID of the blog post or page that it refers to. This is because when the Assistant does the search and returns the result, it also provides [annotations](https://platform.openai.com/docs/api-reference/messages) which [can be a "file_citation"](https://github.com/mikecann/mikecann.blog/blob/main/convex/openai/assistants.ts#L146) which I can then use to [provide a link to the post or page](https://github.com/mikecann/mikecann.blog/blob/main/components/mikebot/MessageContent.tsx#L26).

[![](./link-to-post.png)](./link-to-post.png)

# Conclusion

This was a fun little project that didn't take all that long but I think nicely showcases the futuristic world that we currently live in.

Got feedback or ideas for improvements? Feel free to leave a comment below... or better yet, save us both some time and just chat with Mikebot directly. He's probably more helpful than I am anyway!



I recently took a much-needed vacation to an exotic location. While I was away, I kept a close eye on the [BattleTabs](https://battletabs.com/) servers. Given [our recent global launch](https://mikecann.blog/posts/battletabs-global-launch-on-discord-activities), things were still a bit unstable, and traffic was continuing to increase.

One issue was that, despite all my best efforts, our worker server was still regularly shutting down and not coming back online.

![](./downtime.png)

This generally happened at points of peak load and usually when I was in bed (typical). The result was that chunks of the game wouldn't work or would work in unexpected ways.

Technically, this should not have been possible as Fly.io's machines have [an auto-start/stop feature](https://fly.io/docs/launch/autostop-autostart/) that should ensure that at least one machine is running at all times.

Since I was on vacation and didn’t have time to debug the root cause of the issue, I decided the simplest thing to do was to write a script that would continually check if the worker server was alive, and if it wasn't, start it up.

I knew that this script would need to be online all the time, so I started thinking about how to host it. I could have easily used my favorite tool, [Convex](https://mikecann.blog/posts/im-now-a-convex-developer-advocate), but I had heard about the recently released [Replit Agents](https://docs.replit.com/replitai/agent), so I thought—why not give that a go?

# Replit Agents

If you're not familiar with Replit Agents, here's a quick primer:

<iframe width="853" height="480" src="https://www.youtube.com/embed/IYiVPrxY8-Y" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

The idea was to use their AI to generate my script and host it on Replit. It should be a fairly simple script and, therefore, hopefully a straightforward, isolated task for the AI.

![](./initial-impressions.png)

I hadn’t used Replit before, and my initial impressions were good. I was very impressed with the UI. It felt like a cut-down code editor, but with the agent chat on the side, taking up a large chunk of the UI to let you know that this is a sort of "pair programming" type of workflow.

I started out with what I felt was a pretty clear prompt (see the image above) and was happy to see that the agent was able to easily scaffold out the start of the project with a bunch of files.

After it had generated the files, it asked me if it worked. Unfortunately, it didn’t, so I had to dig through the code and do a bit of guesswork to try to work out what was going on.

![](./env.png)

It probably would have been better if it worked out this issue by itself, but I was impressed to see that they had a little inline UI for adding an environment variable. I thought that was a nice touch.

This is where things started to go off the rails a bit. The AI continued to make a lot of mistakes, both architecturally and logically, which required quite a few prompts on my part to sort out.

Eventually, I had what I thought should work, but I was still getting errors. I eventually tracked down the problem—it stemmed right back to the first prompt, where the AI was using a very outdated endpoint URL for the API.

![](./tracked-down-the-error.png)

After letting it know, it was able to fix the problem, and I finally had a working API call.

As a side note, I was impressed that Replit has a Git integration, and the AI made regular "checkpoint" commits as it went along.

With it working, I now wanted to have it run itself every 5 minutes. This cloud hosting and cron functionality is one of the core features of Replit, so I thought it should be pretty straightforward for the Agent.

![](./always-on.png)

I was super confused at first as it directed me towards an "Always On" feature that doesn’t seem to exist in Replit (anymore). When I pointed it out to the Agent, it even suggested using another service to run the cron!

I had to manually read through the Replit docs before I worked out how to do it myself in the UI.

![](./deployments.png)

With that, my script was up and working, checking the worker every 5 minutes.

# Conclusions

I really like the Replit Agents UI. I think they have done a great job with how it all looks and works. The functionality of Replit itself is also great, and I will definitely keep it in mind next time I need a quick and dirty project.

The AI, however, just makes way too many mistakes right now for me to recommend. Even with my simple script, I had to go in and manually correct it on multiple occasions.

I will, however, continue to monitor how this develops, as this way of developing software projects feels very much like the future direction for coding.



I enjoy reading [Ian Macartney's](https://stack.convex.dev/author/ian-macartney) posts on the Convex blog. When I saw his piece on [Work Stealing](https://stack.convex.dev/work-stealing), a strategy for distributing compute-heavy workloads, I decided to try it myself.

# TL;DR

Check out the demo video below to see what I built:

<iframe width="100%" height="400" src="https://www.youtube.com/embed/sovYE3sWszY" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

Code: [https://github.com/mikecann/whisper-farm](https://github.com/mikecann/whisper-farm)

# Whisper Farm

I aimed to speed up podcast transcription by parallelizing the task into separate "jobs" and performing each in parallel on [Fly.io's GPU machines](https://fly.io/gpu).

Each Fly.io machine runs a single Docker container with a Whisper binary from [whisper-standalone-win](https://github.com/Purfview/whisper-standalone-win), an [FFmpeg](https://github.com/kribblo/node-ffmpeg-installer) binary, and a Node.js "worker" for "chunking" or "transcription" jobs.

To orchestrate this, I used [Convex](https://convex.dev), an excellent backend as a service that I’ve discussed [many times before](https://mikecann.blog/tags/convex).

# Challenges

This project was more challenging than I initially thought.

Most difficulties were related to Docker and Fly.io, while the Convex part was problem-free.

## Docker

Not being a Docker expert caused several headaches when packaging the Whisper model and code.

I started off in high hopes after reading [this blog post](https://fly.io/blog/transcribing-on-fly-gpu-machines/) from Fly. However, I soon realized I didn't want to run it as a standalone container. Instead, I wanted my worker code to run alongside it, simplifying scaling and cost management. I wanted a self-contained worker.

Merging two Dockerfiles turned out to be complex. Eventually, I abandoned that approach and used a [Whisper binary](https://github.com/Purfview/whisper-standalone-win), dynamically downloading and unzipping it when the container was constructed.

Another issue was the large Docker image size, around 15 GB, causing problems on Fly.io. Machines wouldn't start or took a long time, likely due to the image size. Fly.io's error messages weren't helpful here.

To reduce the Docker image size, I learned (thanks, ChatGPT!) that each "RUN" command creates a new layer. By rearranging and combining "RUN" commands, I reduced the image size from 15 GB to 5 GB.

[![](./dockerimagesize.webp)](./dockerimagesize.webp)

This experience taught me that I probably need to learn a bit more about Docker fundamentals.

# UI - Shadcn & Tailwind

Another side quest was experimenting with "the new hotness" in web UI design.

[![](./shadcnui.webp)](./shadcnui.webp)

[Shadcn/ui](https://ui.shadcn.com/) is popular right now. The idea is to copy and paste component code directly into your project and modify it, creating your own component library.

Shadcn uses Tailwind for styling. Although I’ve avoided Tailwind due to its "[stringly typed](https://wiki.c2.com/?StringlyTyped)" nature, I decided to try it since it’s widely used.

My conclusions?

I’m just not sold on Tailwind. I dislike its stringly typed nature and the need for additional tools like `prettier-plugin-tailwindcss` and extra linting rules. Managing className attributes requires more libraries like `tailwind-merge` and `class-variance-authority` which irks me. Tailwind also needs its own compiler/toolchain which is yet another headache to deal with.

Shadcn/ui was also disappointing. While the copy-paste nature is quick, the components lacked features I expected from a UI library. Modifying components that often looked like this is just simply confusing:

[![](./tailwind.png)](./tailwind.png)

# Issues and Future Work

Back to the main point of this post.

I think I proved the concept of work-stealing and learned a lot along the way. There is however more work that would be needed to take this further.

Some key areas include:

- Transcription results might exceed the 1 MB row limit in the database (https://docs.convex.dev/production/state/limits), so storing it as a file might be necessary.
- Error handling for Fly.io machine startup needs improvement.
- More testing is required to ensure stability.
- The code is messy and needs tidying up.
- There’s no authorization or authentication.
- Worker count limits are needed to prevent potential issues.
- The UI is basic and was more of an experiment with Shadcn and Tailwind.



I've been meaning to tinker with Temporal for quite some time now, as it intersects some of my key interests: [Event Sourcing](https://mikecann.blog/posts/serverless-databaseless-event-sourcing), [durable execution](https://mikecann.blog/posts/tech-to-tinker-with-in-2022), and [cool application architectures](https://mikecann.blog/posts/tinkering-with-convex).

# What is Temporal?

From their [website](https://temporal.io/):

> Durable Execution is a development abstraction that preserves complete application state so that upon host or software failure, it can seamlessly migrate execution to another machine.

But what does that mean exactly?

From a coder's perspective, it means you can define "Workflows" in TypeScript such as:

```ts
export async function SubscriptionWorkflow(email: string, trialPeriod: string | number) {
  await sendWelcomeEmailToUser(email);
  await sleep(trialPeriod);
  await sendSubscriptionEndedEmailToUser(email);
}
```

Temporal ensures this function continues to exist even if the server that runs it shuts down. When the server starts up again, it can resume from the correct location, making it "durable." Temporal also handles errors and retries the steps in the Workflow if they fail.

If this doesn't immediately seem impressive, consider how difficult it is to resume a function's execution at any arbitrary point after the fact, especially when dealing with non-deterministic states like DateTimes and random numbers.

# Exploring Temporal

My goal with any "tinkering" I do is to understand the technology enough to form an opinion about it.

I started with [the docs](https://docs.temporal.io/), which are quite thorough. Setting up the environment included installing the Temporal CLI and running their local service, which even comes with a nice WebUI to display your Workflows:

[![](./webui.png)](./webui.png)

After reading through the initial pages of the docs, I summarized the key structure and terms with this diagram:

[![](./diagram.png)](./diagram.png)

Clients start Workflows, which can contain several Activities. Workflows enter a Task Queue from which Workers pull and execute tasks. If activities fail, they can be retried an infinite (configurable) number of times.

Every "[Action](https://docs.temporal.io/cloud/pricing#action)" the system takes (starting a workflow, running an activity, signals, queries, etc.) is an event recorded within the system, which is then recorded against your workflow so you can view exactly what happened and when:

[![](./events.png)](./events.png)

## Event Sourcing

It's worth taking a tangent to discuss another application architecture dear to my heart: [Event Sourcing](https://martinfowler.com/eaaDev/EventSourcing.html).

I won't explain all of Event Sourcing here, as I've covered it before on this blog and there are many good deep-dives available online. The basic idea is that your entire application should be described with immutable events. To get the current state of your application, you fold/reduce your events down to a state. For example, you might have a "UserCreated" event with the user's name "Dave." Later, if Dave changes his name to "Davina," the system triggers a "UserChangedName" event.

Because we record all the events in order and each event is immutable, we can always reconstruct the current state of the user at a later date and see exactly what led to that state. This state reconstruction is called "Projection" in Event Sourcing terminology and is fundamental to Event Sourcing.

Now, suppose you want to model a "Process" like the Temporal Workflow example at the start of this post.

1. A user clicking a button creates a "UserTrialSubscriptionStartRequested" event.
2. A "Process Manager" picks up that event, sends the user an email, and emits a "UserTrialSubscriptionStarted" event.
3. It could also set up a timer so that after the trial period ends, a "UserTrialPeriodEnded" event is fired.
4. The Process Manager detects that event, sends the user another email to let them know their trial has ended, and emits a "UserTrialSubscriptionEnded" event.

What I've outlined with "Process Managers" is exactly what a Temporal "Workflow" does but with a nicer developer experience. Temporal uses clever build-time tooling to ensure that Activities within a Workflow are the entry points when an event occurs. Because side effects are constrained to Activities, we can replay a Workflow to resume a paused state.

# Why?

So why would you want to do this? Many applications (finance, health, military, industry, etc.) need to ensure something happens and have a clear record of exactly what happened to potentially reconstruct events.

Event-based architectures like Event Sourcing and Temporal capture the system as events, then reconstruct the world from those events, rather than the other way around (a typical CRUD system). Temporal.io cleverly combines the best parts of Event Sourcing (the Process Managers) into an entirely standalone product and names it Durable Execution.

# Unanswered Questions

The Temporal docs are extensive, and there's a lot I haven't explored yet.

I've run out of time for this tinkering session, but next time I'd like to build a simple application to see what works and what doesn't. Here are some notes for next time:

- How does input validation on Workflows work? Is there input validation at all?
- Signals and Queries look interesting but a bit convoluted—need to explore those more.
- Should you use Temporal for your entire application state or just Workflow-based state? If not, how do the two combine?

# Pricing

Temporal uses a per-Action-based pricing model:

[![](./actionpricing.png)](./actionpricing.png)

One million "actions" costs $25. This seems quite expensive since everything results in an action. For comparison, [Convex](https://www.convex.dev/pricing) charges $25 per month for 25 million function calls, and [Cloudflare](https://developers.cloudflare.com/workers/platform/pricing/) charges $0.3 per 1 million function calls.

However, I'd need to make a real app to get a proper sense of how expensive Temporal would be in practice.

# Thinking in Convex

Speaking of Convex, I can't help but think about how I might implement a "Workflow-like" structure on Convex.

I've been considering building an Event-Sourced application on Convex for a while, but I think a simpler structure via State Machines might achieve most of what you want... 🤔

Stay tuned—I may well tinker with this in the future!



As is [rapidly becoming a trend](https://mikecann.blog/posts/tinkering-with-spacetime-db) in my tinkering adventures, today's post chronicles a bit of a failed adventure. But that's okay, it's always good to explore and learn, particularly in the rapidly evolving field of AI.

# LazyAI

There are a lot of these text-to-app startups around at the moment. One of the most famous is [Devin from Cognition.ai](https://www.cognition.ai/introducing-devin). That one is not available yet, but I wanted to explore another one that has been getting some buzz lately: [Lazy.ai](https://www.getlazy.ai/).

[![](./tagline.png)](./tagline.png)

Its tagline is "Idea to software 10X faster." Let's see how it does.

# The Project

As I am [a massive Convex.dev fan](https://mikecann.blog/posts/im-now-a-convex-developer-advocate), I knew it had to be something Convex related. I felt like this could be tough for the AI, as Convex is not as well-known yet, and so the LLM may not have as much pre-trained knowledge.

Something I had wanted to experiment with for a while was some sort of visualization for Convex database schemas so you can see the relationship between tables and types. I was imagining some sort of class diagram:

[![](./classdiagram.png)](./classdiagram.png)

Or even better, a program that can generate the diagram dynamically such as [TSDiagram](https://tsdiagram.com/).

[![](./tsdiagram.png)](./tsdiagram.png)

# LazyAI Conversation

So, with that, I entered the following prompt into LazyAI:

> I would like to build an app that takes in a convex database schema and generates a Directed Acyclic Graph of the relationships between the tables and types.

It came back with a helpful plan of attack.

[![](./lazy1.png)](./lazy1.png)

I felt like maybe I was being a bit tough on it, so I gave it a bit more info:

> My project is TypeScript based. I want to be able to run this tool as a TypeScript script and have it analyze the convex.dev database schema that I give it. For example:

```ts
const schema = defineSchema({
  messages: defineTable({
    authorId: v.id("users"),
    body: v.string(),
    likes: v.int64(),
    replies: v.int64(),
    isReplyToMessageId: v.optional(v.id("messages")),
    imageId: v.optional(v.string()),
  }).searchIndex("search_by_body", {
    searchField: "body",
    filterFields: ["authorId"],
  }),

  users: defineTable({
    handle: v.string(),
    name: v.string(),
    pictureUrl: v.union(v.string(), v.null()),
    tokenIdentifier: v.string(),
  })
    .index("by_token", ["tokenIdentifier"])
    .index("by_handle", ["handle"]),

  lists: defineTable(
    v.union(
      v.object({
        kind: v.literal("all_messages"),
        ownerId: v.id("users"),
        name: v.string(),
      }),
      v.object({
        kind: v.literal("search"),
        ownerId: v.id("users"),
        name: v.string(),
        query: v.string(),
        includeReplies: v.boolean(),
      })
    )
  ).index("by_ownerId", ["ownerId"]),

  likes: defineTable({
    likerId: v.id("users"),
    messageId: v.id("messages"),
  })
    .index("by_likerId", ["likerId"])
    .index("by_messageId", ["messageId"])
    .index("by_messageId_likerId", ["messageId", "likerId"]),
});
```

The response it came back with was:

[![](./lazy2.png)](./lazy2.png)

I felt like that was actually a reasonable plan and told it to go ahead.

Now, the way the app works is that it's split into three parts: Test, Code, and Env Secrets.

[![](./lazyparts.png)](./lazyparts.png)

So in theory, I should be able to test the output it generates. Well, this is where things unfortunately start to go off the rails a bit because to test (even your first project), you need to sign up for a pro membership.

[![](./lazypro.png)](./lazypro.png)

Which I would have been happy to do if I felt like the tool was going to give me some value, but the pricing scared me off.

[![](./lazypricing.png)](./lazypricing.png)

$39.89 per month just to see if the code works and then usage costs on top of that?! No chance I was going to pay that just to check it out.

So after a little bit more back and forth with the AI, I decided to take a look at the code and copy it into VSCode to see if I could get it to run myself.

Well, unfortunately, I was to be disappointed again.

[![](./lazyerrors.png)](./lazyerrors.png)

There was no chance this code would compile or even run. There are TypeScript type errors all over the place and the imports wouldn't work from the wrong packages, etc.

I tried to give it the benefit of the doubt though and fixed the compile errors and import errors and tried to run it. Sure enough, no dice.

[![](./lazyrunerror.png)](./lazyrunerror.png)

# LazyAI Conclusion

It's not ready yet. I think the task I gave it was quite tricky and maybe I was asking too much, but I think I was willing to give it the benefit of the doubt until I saw the pricing for it. Then my sympathy evaporated.

Fortunately, there are a lot of these "AI Project Generation" projects out there and as I still had a bit of time left to tinker, I thought I would give another one a go.

# Pythagora.ai

https://www.pythagora.ai/

[![](./pythagora.png)](./pythagora.png)

This one touts itself as a "dev tool that builds apps from scratch by talking to you." Sounds great, let's give it a shot.

This one works a little differently. It is a VSCode extension that you need to install [from the store](https://marketplace.visualstudio.com/items?itemName=PythagoraTechnologies.gpt-pilot-vs-code&ssr=false#review-details). So I headed over there and immediately was discouraged to see that it was only 3 stars out of 5. I was not off to a good start.

[![](./pyreviews.png)](./pyreviews.png)

After a bit of a confusing setup (you have to first install another tool, gpt-pilot apparently) I had it working.

[![](./pyworking.png)](./pyworking.png)

(just a side note, having an AI running on my local system from an unknown developer with full access makes me kind of nervous.)

This time I was a little more generous with the initial prompt:

> I would like to build a Node.js TypeScript app that takes in a convex.dev database schema and generates a graph of the relationships between the tables and types.
>
> I want to be able to run this tool as a TypeScript script inside of my convex project and have it analyze the convex.dev database schema that I give it.
>
> I want the output to be an image that looks like something like a class diagram from classic UML.
>
> This is an example convex schema:

```ts
const schema = defineSchema({
  messages: defineTable({
    authorId: v.id("users"),
    body: v.string(),
    likes: v.int64(),
    replies: v.int64(),
    isReplyToMessageId: v.optional(v.id("messages")),
    imageId: v.optional(v.string()),
  }).searchIndex("search_by_body", {
    searchField: "body",
    filterFields: ["authorId"],
  }),

  users: defineTable({
    handle: v.string(),
    name: v.string(),
    pictureUrl: v.union(v.string(), v.null()),
    tokenIdentifier: v.string(),
  })
    .index("by_token", ["tokenIdentifier"])
    .index("by_handle", ["handle"]),

  lists: defineTable(
    v.union(
      v.object({
        kind: v.literal("all_messages"),
        ownerId: v.id("users"),
        name: v.string(),
      }),
      v.object({
        kind: v.literal("search"),
        ownerId: v.id("users"),
        name: v.string(),
        query: v.string(),
        includeReplies: v.boolean(),
      })
    )
  ).index("by_ownerId", ["ownerId"]),

  likes: defineTable({
    likerId: v.id("users"),
    messageId: v.id("messages"),
  })
    .index("by_likerId", ["likerId"])
    .index("by_messageId", ["messageId"])
    .index("by_messageId_likerId", ["messageId", "likerId"]),
});
```

Then followed a plan that the AI came up with and interestingly, it asks me before it runs commands:

[![](./pycommands.png)](./pycommands.png)

Ughh, it seems like it gets stuck quite easily though when something isn't installed that it needs:

[

![](./pydot.png)](./pydot.png)

I didn't want to install anything else right now, so I just clicked past it.

It seemed to continue just fine, then began installing a bunch of Node packages and setting up a TypeScript project for me, which was kind of nice.

It stops every now and then to check with you to make sure everything is looking good, which is nice.

[![](./pyreview.png)](./pyreview.png)

It started to generate some TypeScript. It looked like there were some errors, but I was starting to feel confident that it might be able to debug its way out of it. Then I hit:

[![](./pyfreetrialover.png)](./pyfreetrialover.png)

Fortunately, their pricing is much more reasonable than LazyAI and even lets you use your own OpenAI keys!

[![](./pyapikey.png)](./pyapikey.png)

I was keen to see where this went, so I decided to spend $10 (and tip the developer) to find out.

I put together this little video to give you a sense of what the process feels like:

<iframe width="853" height="480" src="https://www.youtube.com/embed/YmhPVFl-dVE" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

Unfortunately, as can be seen from the video, the code is very broken and would not run even if I followed the instructions they gave me.

# Conclusion

Unfortunately, I had now run out of time to tinker with this any more, but I felt like it was possible that I could have gotten to something usable with Pythagora, which I didn't think was likely with LazyAI.

Could I build a graph visualization faster myself? Probably, but that's not the point. The point is that I can potentially hand this task off to my "junior developer" and have them work on it for me while I do something else, like touching grass for example (ewww).

So my takeaway is that it's not ready yet, at least these two aren't ready.

LazyAI has a long way to go, but I think I do prefer the running code in an online sandbox process over Pythagora's run the code on your local system approach, which quite frankly makes me nervous.

Running in a sandbox also means that the AI doesn't have to continually pause to ask me if it's okay to run a command (like Pythagora does), as it can simply run it without fear of damaging the underlying operating system.

I am looking forward to future versions of App Generators. In particular, [Devin](https://www.cognition.ai/introducing-devin) looks very promising and also runs in a sandbox-like environment. I added myself to their waitlist, so stay tuned. I may well revisit this in a future blog post!


I have got to be honest, I have never heard of [Web Dev Cody](https://www.youtube.com/@WebDevCody) before but when I saw that he had a [hackathon](https://hackathon.webdevcody.com/) in partnership with [Convex](https://www.convex.dev/) I knew I was going to have to carve off a bit of time from [BattleTabs](https://battletabs.com/) development to have a crack.



# Competition Categories & Ideation

[![](./categories.png)](./categories.png)

The competition was themed around Collaboration and AI so I racked my brain for a bit before coming up with an idea.

I wont spend too many words explaining what I built, instead ill let the video that I used to demo the project do the talking:

<iframe width="853" height="480" src="https://www.youtube.com/embed/XZ2iCEcW_io" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

# Results

Out of [53 submissions](https://hackathon.webdevcody.com/submissions), I came third in the AI category which im pretty happy with considering I spent less than a day on it.

Watch Cody talk about my project here:

<iframe width="853" height="480" src="https://www.youtube.com/embed/PJjk4Qg_GiU?si=9oUEhACACbj3KxkR&amp;start=186" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

# Conclusion

I had a ball working on this, its pretty astounding that you are able to build this much functionality in such as short time. I was able to implement and iterate as fast as I could go without any boiler-plate bogging me down.

[Convex](https://www.convex.dev/) really is an incredible technology, if you are looking to do a web app you should definitely check it out!

If you are interested in playing with Chatter Craft yourself the demo is here: https://chatter-craft.netlify.app/

If you want to peek under the covers a bit the source is available here: https://github.com/mikecann/chatter-craft


AI is all the rage at the moment and for good reason, its going to revolutionize many industries.

Since I first wrote about it in the context of Game Development back [in November](https://mikecann.blog/posts/ai-and-the-future-of-game-development) I have been thinking how we can apply some of the advances to [our](https://gangbusters.io/) game [BattleTabs](https://battletabs.io/).

# Problem

Since the first version of the game (3+ years ago) we allowed users to name their player. This, we feel, gives a better sense of ownership and attachment to the game.

[![](./signup.png)](./signup.png)

A year or so after launch [we introduced the "Custom Fleets" feature](https://mikecann.blog/posts/battletabs-ships-and-fleets-update). As part of this feature we also allowed the user to name their fleet.

[![](./fleet-name.png)](./fleet-name.png)

Again we feel this creates a stronger feel of attachment to the game and their choices within it.

Both of these features however opened up an annoying issue for us: profanity.

We started to get reports of players being offended by the names of other players.

[![](./early.png)](./early.png)

This was an issue as lots of children played BattleTabs so we felt like we had to do something about it.

# Local Filtering

So.. we started to implement a "naughty words" list.. which grew.. and grew and grew..

[![](./list.png)](./list.png)

No matter how many words or phrases we put on this list, users always seemed to find ways to be offensive.

Did you know for example that "Tangina nyo" is a swear word in Fillipino? Nope, neither did I.

# Community Filtering

So after many months of this we decided to enlist some help.

We implemented a "Report Player" button on the player profile page.

[![](./report.png)](./report.png)

This then sent a "Report" to a special channel in discord that only us and our Community Moderators could see.

[![](./reports.png)](./reports.png)

We then added special discord commands to our bot that allowed our moderators to rename the player and then potentially ban then from renaming themselves again (repeat offenders).

This worked.. kind of.

Obviously it relied on people reporting others and then out Community moderators being available to handle the report.

So it meant that some names slipped through the cracks or went days before being detected.

# AI Enters The Chat

Enter ChatGPT in late 2022 and suddenly every problems seems possible to solve with the hammer that is LLMs.

So we decided to do some quick experimentation and see if AI could detect profanity in a phase.

We created a sample spreadsheet of a sample of profanities and non-profanities and hooked up ChatGPT plugin to Google Sheets and tried out a few prompts.

[![](./sheet.png)](./sheet.png)

It definitely _seemed_ to work. Enough that it was worth going ahead and implementing it in the code.

Ill save you all the iterations we did around the prompts and just show you what we ended up with:

```ts
{
  max_tokens: 20,
  temperature: 0,
  messages: [
    {
      role: "system",
      content: `I want you to act as a profanity checker. I will give you potentially profane word or phrase and you will tell me if it is profane or not. Do not provide personal opinions or assumptions. Do not respond asking how you can assist. Simply respond with the words 'profane' or 'not profane'. If it is profane you may also give a very short explanation of why you think it is profane.`,
    },
    {
      role: "user",
      content: `the phrase is: '${phrase}'`,
    },
  ],
  model: "gpt-3.5-turbo-0301",
}
```

We set the temperature to very low as we wanted it to give consistent outputs given the same inputs, but im not sure is that makes much difference.

We have quite a long prompt, this was based on recommendations from [awesome chat gpt prompts](https://github.com/f/awesome-chatgpt-prompts).

We also added the `the phrase is: '${phrase}'`, instead of simply providing the phrase because for some reason the AI would still get confused and try to respond to the phrase rather than the system message:

[![](./whoops.png)](./whoops.png)

For example in the above you can see that if the player provides their name as "yes" then the AI responds with "Great, let's begin. What is the first word or phrase you would like me to check?" instead of the expected "not profane".

Its not perfect and we are still refining it but it generally works really well and has caught thousands and thousands of profane player names and fleet names.

[![](./results.png)](./results.png)

# Conclusion

In the end im super happy with how this has turned out. For minimal effort and cost we were able to get a much more super profanity checker improving the experience for the 99.9% of our players who dont want to see profanity while playing [BattleTabs](https://mikecann.blog).



(no this isn't an april fools post :P)

I wrote my first Application 31 years ago when I was [7 years old](https://mikecann.blog/about). Since then I have spent most of my career learning the fine art of of crafting clean code. From hand-crafted HTML and CSS to wrestling with MXML and AS3 I have been doing this for a long time.

With the incredible advances in Large Language Models (LLMs) and their ability to generate code on demand i'm starting to wonder if the days of hand-crafted applications are numbered.

# What is a LLM?

You must have been living under a rock if you haven't heard of ChatGPT over the last few months, but just in case you are a granite hermit:

> A LLM is a large neural network that has been trained on a large corpus of text. The most famous of these is [GPT-3](https://openai.com/blog/openai-api/) which has been trained on 175 billion words of text. The idea is that you can feed it a prompt and it will generate text that is similar to the prompt. For example, if you feed it the prompt "Describe the life and work of Albert Einstein", it will generate a text that provides information and details about Einstein's life and contributions to science.

(yes the above was generated using a LLM)

# What is an Application

For the purposes of this post, let's say that an application is made up of three parts:

1. The persisted state or "data"
2. The interface into that data (API). This is generally your "server".
3. The User Interface that makes use of the API. This is generally your "client"

I think that each of these parts is going to undergo a massive transformation thanks to the recent AI advances.

As its most familiar to most people lets start with the UI.

# What is a UI?

A UI is a User Interface. It is the thing that you interact with to use an app or program. Almost every app has one, think excel or discord:

[![](./excel.png)](./excel.png)

The only real examples I can think of that don't are things like CLIs (Command Line Interfaces), but even then its arguable if a CLI even if its just text based.

One way to think about them is they are the visual front end on whatever business logic and data that makes up the core of you application. It allows you to query and mutate that data in a way that is easy for the user to understand.

# UIs of Today

Current UIs are generally hand-crafted by humans to look and feel a certain way. These usually are a very labour intensive pursuit on the behalf of the developer.

If you don't like or understand the particular design of a given UI, you are generally out of luck because that's the way it's been developed.

# UIs of Tomorrow

Back at the start of [November 2022](https://mikecann.blog/posts/ai-and-the-future-of-game-development) I wrote a post on how I thought AI would change the future of game development. In that post I talked about how I thought that the future of app development will feel more like sculpting than coding.

> I have spent so much of my career thinking about and worrying over the best practices for programming. Am I correctly implementing the SOLID principles in OO C# or and I fully understanding the Monad or ADTs in Functional Programming? In the future IT DOES NOT MATTER. The output is the only thing that matters.

> I think about the future of programming more as sculpting. You start off with a blank slate then using your own words you command by command iterate towards your end goal.

This was only five short months ago, but so much has happened since then. ChatGPT was released and became [the fastest growing user base of all time](https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/).

The world suddenly realized that you can get a long way with just a text-based conversation via a chat window

This revelation caused [Microsoft to double down on their investment into ChatGPTs creator OpenAI](https://www.afr.com/technology/microsoft-invests-14b-in-chatgpt-maker-openai-20230124-p5cex0) and to very rapidly [begin implementing chat-like interfaces](https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/) into all their products.

You can now simply ask an assistant for information using natural language, and it can go out to the web, find that information, summarize it, and contextualize it within a few seconds. This has many questioning why would you ever need to use a search engine?

Well, this combined with the double-pronged threat of vastly reduced cost of producing generated fake but believable content, caused Google to [declare a "Code Red"](https://www.forbes.com/sites/davidphelan/2023/01/23/how-chatgpt-suddenly-became-googles-code-red-prompting-return-of-page-and-brin/?sh=720f5a2b5977) to [frantically rush](https://www.reuters.com/technology/google-ai-chatbot-bard-offers-inaccurate-information-company-ad-2023-02-08/) to catch up.

These new new LLMs aren't just good at generating text however, they are also really good generating code too.

So what has all this got to do with User Interfaces? Well if you put it all together im imagine a world where an application is simply its data and an API. The User Interface is generated on demand by a LLM.

For example take a look at Microsoft's recent [release of Security Copilot](https://www.reuters.com/technology/google-ai-chatbot-bard-offers-inaccurate-information-company-ad-2023-02-08/):

<iframe width="560" height="315" src="https://www.youtube.com/embed/g1HoXNoP3V0" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

The UI for Security Copilot is simply a chat prompt. The has a conversation using natural language to get the information they need.

[![](./security-copilot.png)](./security-copilot.png)

The requested information is returned and decorated using Microsoft's own user interface controls and styling.

I believe this is just the start of what is going to happen.

Imagine instead of simply returning the information requested, the AI that lives inside your application will also return some UI code that can be used to render that information.

It's important to note that this code might be made up of building blocks that the developer has specified so that the application retains the same look and feel. This however isnt entirely neccessary as the LLM will know how to generate a good UI to represent the information that it contains.

In this future applications become much more personal and are tailored to the way you want them to work.

Sure there are going to be occasions where you just want to get the experience that the developer intended, I can imagine there might be some pre-defined layouts but then you are free to customize it to your hearts desire.

We [at Gangbusters](https://www.gangbusters.io/) use the excellent tool [Metabase](https://www.metabase.com/) to query and visualize our data.

[![](./metabase1.png)](./metabase1.png)

Its a fantastic tool with a great interface for "asking questions" of your data and then visualizing the results.

[![](./mb-questions.png)](./mb-questions.png)

But it could be better, what if you could just use natural language to ask those questions and instead of simply showing a predefined pie chart or graph, instead it could tailor the UI to whatever best suits the data. It could provide buttons and controls to allow you to manipulate the data.

This general thing is how I imagine application UIs will work in the future.

If we take it to its logical conclusion then perhaps the operating system's UI itself is generated dynamically on demand. There are no "applications", the AI that runs in the OS simply goes out to the various services on behalf of the user, returns with the data and generates a UI to display it on the fly.

# The Future of APIs

This is a bit of a tangent but:

UIs need to render some data to the screen. They get that data from a data source that they interface with using an Application Programming Interface (API).

A normal website is a UI that is communicating with one or many servers via an API generally REST or GraphQL.

These APIs are generally written by humans so that they are easy to understand by other humans.

In the age of generated UIs I think this will change. APIs no longer need to be written for human consumption primarily they can instead by written for machine consumption. This will mean a lot less work for the developer that has to create and document an API for other humans to consume.

Perhaps the API will simply be a text prompt. You will in natural language simply ask for the data and then an AI that lives in your application will understand that, convert it into an SQL statement and then return the data to the UI.

[![](./sql.png)](./sql.png)

We can add "system" prompts to our AI to ensure that it doesnt return any private data or allow it to manipulate data in any undesirable ways.

# The Future of Data Storage

So you have an AI generating your UI and the API is simply a prompt, whats left? Well just the data storage part.

In the section above I imagine a future where an AI is able to generate an SQL query to interrogate an existing relational datasource such as Postgres. But what if we took that one step further.

What if we wanted the data storage to be more flexible so that as the application evolves the new structure of the data can be stored correctly?

Currently we have to define migrations to modify the schema of our database as the application evolves over time but what if instead data was stored in a less-structured NoSQL like manner. Now your data storage can change over time without and migrations.

The AI can still write NoSQL queries to access the data and it could also record a "dynamic" schema that it uses before it queries the data source.

Now if we take that another step forward, what if the AI is able to run in the background and spot inefficiencies in the way the data is structured and restructure it on the fly?

In this world you no longer know how your data is stored as the AI has organized and evolved it as your application has evolved.

But we can take this a step further. What if your data isn't structured at all? This is a future suggested [by @DYtweetshere in January](https://twitter.com/DYtweetshere/status/1617471632909676544?s=20).

[![](./backend.jpg)](./backend.jpg)

Instead the the AI "backend" simply "remembers" all the previous prompts ever given and is able to infer structure and meaning from that data as new prompts come in.

This is the ultimate in storage flexibility as there is now no longer and structure to your data, it is instead simply a list of strings.

Now this final example will take a few more advances in model context size as the AI must keep everything ever muttered in its "memory" so its can draw meaning from it without querying it. For a large app this is currently infeasible, but I suspect possible in the future.

The other potential is it stores a "compression" of the data via weights in exactly the same way the AI itself is trained but that by its very nature would be a lossy mechanism and thus your stored "Todos" (for example) would only have a probability of being recalled exactly which might not be ideal.

# Conclusion

For the past 31+ years I have been making applications in generally the same way. Hand-crafted some code for the UI, API and Data store, refine and repeat. I think this age is finally coming to an end.

I think the next few years are going to see some of the biggest changes in the way applications are created and consumed. More so than the cloud computing revolution, more so than even the mobile computing revolution.

I for one am excited about the potentials for these changes and look forward to what the future brings!



As a [game developer of 30](/about) years I like to stay up to date with new tools and techniques in my chosen profession. There is however a change coming of such magnitude I think its going to totally upend the way games are made in the future.

# History

Before we can talk about the future of Game Development we must first take a quick trip to the past to see how we got here.

I remember first being blown away by AI powered image generation technology 6-7 years ago when [CycleGAN](https://github.com/junyanz/CycleGAN) was first released allowing you to "Style Transfer" a horse into a zebra:

[![](./horse2zebra.gif)](./horse2zebra.gif)

This led to a whole world of incredible advances, notably [StyleGAN](https://arxiv.org/abs/1912.04958) brought the ability to generate very realistic human portrait photos. Checkout this page and hit refresh a few times:

[https://thispersondoesnotexist.com/](https://thispersondoesnotexist.com/)

This then led to an explosion of [This X does not exist](https://thisxdoesnotexist.com/) websites that are all based on GANs (Generative Adversarial Networks).

Around that same time there was a whole wave of "Deep Fakes" that used the StyleGAN process to impose peoples faces onto video creating incredibly realistic videos that could very easily trick unsuspecting viewers.

<iframe width="853" height="480" src="https://www.youtube.com/embed/gLoI9hAX9dw" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

Up to this point most of these techniques had been limited to either transferring style or generating images based on a highly specialized model.

Then came along a new wave of generic language models based on [Google Brain's Transformer Networks](https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html), notably [GPT-2 and 3 from OpenAI](https://beta.openai.com/examples/).

[![](./gpt3.png)](./gpt3.png)

These massive multi-billion neuron models understand language much more deeply than anything that had come before them and are able to understand and translate human language with incredible accuracy.

Don't tell my wife but I actually used GPT3 to help me craft my wedding vows:

[![](./vows.png)](./vows.png)

These advances in language understanding were critical to the next wave in image generation; text to image.

You see, to be able to generate images from some text that you provide then the AI must first deeply understand all the words in a given prompt.

For example lets say we want "An oil painting of a cat surfing a massive wave while riding on a lemon".

To accomplish this the AI must have a deep understanding of not only what a cat, a lemon, wave and surfing are but it must understand those words in the correct order and then understand how to put them together to look like an "oil painting".

[![](./lemon-cat.png)](./lemon-cat.png)

[OpenAI's Dall-e-2](https://openai.com/dall-e-2/) was released this year as a commercial product was used to generate the above images.

Arguably more impressive images are able to be generated by [Stable Diffusion](https://stability.ai/blog/stable-diffusion-public-release) which unlike Dall-e-2 was released for free to the general public:

[![](./lemon-cat-sd.png)](./lemon-cat-sd.png)

Stable Diffusion and its cousin [Mid Journey](https://www.midjourney.com/showcase/) are more targeted towards "Concept Art" than Dalle-e-2 this means that althought it might still understand the words "cat", "lemon", "surfing" when it comes to generating the images however it is less likely to do a good job on the "oil painting" part of the prompt than Dalle-e.

When given a task that its designed for however it is REALLY good. I mean just checkout some of the examples on this page with the prompts used to generate them: [https://openart.ai/discovery?dataSource=md](https://openart.ai/discovery?dataSource=md)

Designing the correct prompt is itself an art however and there are now job positions opening for "Prompt Engineers".

# Game Development

So what has all this got to do with the future of game development?

## Concept Art

Well firstly in my opinion Stable Diffusion and Mid Journey have all but destroyed the "Concept Artist" sub profession in game development.

A bold statement I know, but why would you pay hundreds of thousands of dollars per year for a highly specialized concept artist when in a few seconds you can generate images like this:

[![](./fortnite.png)](./fortnite.png)
["fortnite in the year 2060"](https://openart.ai/discovery?dataSource=md&q=game)

[![](./house.png)](./house.png)
["medieval fantasy monster tavern inn, hidden mimic monster living organic building , isolated lane in mountains, deep forest, winding paths, lonely traveller, detailed living organic architecture hidden monster, Tudor fantasy historical, eye-windows, fangs, mouth-doors, tongue, rain, digital concept art, menacing"](https://openart.ai/discovery?dataSource=md)

[![](./woman.png)](./woman.png)
["beautiful cyberpunk woman in armor, dark hair, futuristic, intricate binary and electronic designs, elegant, highly detailed, sharp focus, ghost in the shell, mamoru oshii, nier automata, art by wlop, art by guweiz"](https://openart.ai/discovery?dataSource=md)

Sure, they aren't perfect, so I guess you could make the argument that concept artists in their current form would still be needed, if only to tweak the output to the intended application.

I suspect its more likely that "a concept artist"'s job will change. Instead of pushing pixels around with a [Wacom tablet](https://www.wacom.com/) instead they will simply become "Prompt Engineers". Taking in the request from their client and using their experience with a number of specialized models to iterate towards a result.

## Generalist Artist

Right now the models arent particularly well tuned for generating sprites / textures or other assets that we currently use to build games but I suspect that is either currently being worked on or is currently being used in the larger studios.

Given tools such as [Pixel Diffision](https://colab.research.google.com/github/KaliYuga-ai/Pixel-Art-Diffusion/blob/main/) or [Pixelator](http://pixelatorapp.com/) or [Monster Mash](https://ai.googleblog.com/2021/04/monster-mash-sketch-based-tool-for.html) I suspect its just a matter of joining the dots and putting these pieces together in one cohesive package.

Maybe [even more recent](https://waxy.org/2022/11/invasive-diffusion-how-one-unwilling-illustrator-found-herself-turned-into-an-ai-model/) advances from [DreamBooth](https://dreambooth.github.io/) will allow us to train models designed specifically for our game's art style.

Having said that I think ultimately we wont have such low-level tooling that humans use, instead, I believe things will go much grander direction..

## Code

So art in Game Development as we know it is dead. Surely we will still need coders like myself. Right? The 30 years of game-making experience I have built up aren't simply going to become obsolete. Surely?

Github's [Copilot](https://github.com/features/copilot) was released last year to much fanfare and its easy to see why. Given a simple prompt written in a comment it's able to generate large swaths of working code for you:

[![](./copilot.png)](./copilot.png)

As with image generation it uses very large complex language models under the hood to understand to break apart your prompt (and other context specific information from your workspace) and transform that into an output, generating one token at a time.

OpenAI, the creators of GPT-3 have also been working on their own code generation tool they call [Codex](https://openai.com/blog/openai-codex/). This I think this strongly hints at the direction things are going to go for game coders of the future:

<iframe src="https://player.vimeo.com/video/583550498?h=90927a1846&color=ffffff&title=0&byline=0&portrait=0" width="640" height="360" frameborder="0" allow="autoplay; fullscreen; picture-in-picture" allowfullscreen></iframe>

See how the programmer no longer writes code. Instead he describes iteratively what he wants and lets the AI generate the code.

What happens in this world where the programmer no longer needs to write code? The syntax for the code becomes your own language; English, French, Chinese whatever. The countless arguments over semicolons or braces become obsolete and irrelevant when you no longer have to write using syntax dictated by a "programming language".

I have spent so much of my career thinking about and worrying over the best practices for programming. Am I correctly implementing the SOLID principles in OO C# or and I fully understanding the Monad or ADTs in Functional Programming? In the future <b>IT DOES NOT MATTER</b>. The output is the only thing that matters.

I think about the future of programming more as sculpting. You start off with a blank slate then using your own words you command by command iterate towards your end goal.

[![](./sculpt.png)](./sculpt.png)
"a person sculping a half-finished masterpiece from a block of marble with a chisel, background is the inside of a grand building"

Imagine a future IDE / Game Engine such as [Unity](https://unity.com/) or [Unreal](https://www.unrealengine.com/) where you write no code, there is no code to interrogate. All there is is a prompt. Sure you can still select things but that is only to give context to the prompt rather than to directly edit its vertices or textures or anything like that.

[![](./mockup1.jpg)](./mockup1.jpg)

# Conclusion

Do I think this is going to happen tomorrow, next year, next 5 years? Probably not but the more I think about it the more I see it as an almost inevitability that this is the direction things are going to go. Particularly as we integrate more and more AI into our work lives.

Don't believe me? Check out this incredible Tweet from Nat Friedman who used GPT-3 to generate an intelligent web-crawling agent to automate browser tasks: https://twitter.com/natfriedman/status/1575631194032549888?s=20&t=3eSi6pFWeSE7rMui4kdyyA

His prompt and generated code can be found here: https://github.com/nat/natbot/blob/27a357115093cfe9ca927c9e22fd07048e91eb36/natbot.py#L15

Now imagine that you couldnt see the code output python code, it doesnt matter. Just so long as the final product works as intended who cares if the code "looks good" or not.

I guess in some aspects this is the ultimate promise of [BDD](https://en.wikipedia.org/wiki/Behavior-driven_development). We specify in written english what the system under test should do and dont care about how it does it so long as it does it.

Perhaps BDD will become the programming "language" of the future?

```
Title: Mario 2032

As a game player,
I want to play a side scrolling platformer where I control a player that has to run and jump over obstacles and avoid enemies and solve puzzles

Scenario 1: The player can move left or right
Given that the player is alive and is controllable,
When I press left or right on the keyboard,
Then I should move the character left or right depending on the direction pressed.
```

Keen to hear your thoughts, drop me a comment below!


For the past 3 years (almost) I have been working as a software developer at TapSlots in Perth Australia. Its been a wild ride of learning and development but over the Christmas break I decided that its about time that I tried something new.



For some time now I have been very interested in Machine Learning and AI. I have been thinking about this ever since reading my first sci-book Isaac Asimov. I, Robot. At 18 I chose not to pursue a university degree in Robotics and AI and instead took my path down a games development route instead.

General purpose AI has always seemed an elusive technology always just outside the grasp of us hairless monkeys. Like fusion power, or a duke-nukem sequel it seems like we are continually just out of reach of the singularity. Recent advancements in deep machine learning however have made it seem that maybe, perhaps maybe we might be getting close.

CNNs, LSTMs, GANs and other ML acronyms de jour are making incredible strides in very specific problems. These arent solutions to the General Purpose AI problem but with them comes incredible interest and funding.

From an entrepreneurial perspective, this sudden surge in AI advancements has brought incredible opportunities for disruption of existing businesses. When combined by cloud powered IoT I dont think there is a single industry that wont be transformed over the next decade or two.

So when I was trying to decide were I should focus my most valuable asset, time, next, I concluded that it was obvious that I should be pointing myself in the direction of AI.

Fortuitously around the same time I was approached to join a new startup in Perth that was using Deep Learning to solve some very interesting computer vision problems for the retail industry. After thinking about it some more I decided that this was an exciting opportunity and aligned well with aspirations and beliefs.

So as of this week I am now the Head of Engineering at GenVis in Perth Australia. Im really excited for the potential of this company and I hope that I can do what I do best, explore and advance new technologies, to bring success in my new role.

In my [last post](/machine-learning/a-game-developer-learns-machine-learning-a-little-deeper/) we dug a little deeper into machine learning by starting from scratch and training an agent on a vastly simplified version of mr nibbles. In this post we will explore what happens when we step it up a notch and attempt to train on a significantly more complex version of the game.



# The Series

If you are new to the series I recommend you checkout the previous posts first, here's the whole series:

0. [A Game Developer Learns Machine Learning - Intent](/machine-learning/a-game-developer-learns-machine-learning-intent/)
1. [A Game Developer Learns Machine Learning - Getting Started](/machine-learning/a-game-developer-learns-machine-learning-getting-started/)
1. [A Game Developer Learns Machine Learning - A Little Deeper](/machine-learning/a-game-developer-learns-machine-learning-a-little-deeper/)
1. A Game Developer Learns Machine Learning - Mr Nibbles Basics

# Code

If you want to see the code for this post then checkout this tag: https://github.com/mikecann/MrNibblesML/tree/MrNibblesBasicsBlogPost

# Mr Nibbles Forever vs Mr Nibbles

My original [Intent](/machine-learning/a-game-developer-learns-machine-learning-intent/) was to teach an ML agent to learn my game [Mr Nibbles Forever](http://epicshrimp.com/app/mrnibblesforever/). Based on my experience up to this point I think what might be easier is to begin with trying to train an agent to learn a game similar to "[Mr Nibbles](http://mr-nibbles.com/)" instead of "[Mr Nibbles Forever](http://epicshrimp.com/app/mrnibblesforever/)".

<iframe width="853" height="480" src="https://www.youtube.com/embed/lyAf7VVLdKg" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

Rather than an endless procedurally generated game like Mr Nibbles Forever, Mr Nibbles is a level-based platform-puzzler.

I suspect that Mr Nibbles will be easyier to learn as it's significantly less complex than its later variant, Mr Nibbles Forever.

# The Environment

Before I can even think about training my agent, I need an environment. Mr Nibbles was originally made many years ago in HaXe and thus I will need a way to build levels in Unity.

Unity recently launched a [set of tools](https://docs.unity3d.com/Manual/Tilemap-Painting.html) for generating tile-based games.

So I set to work using those tools to construct some levels:

<iframe width="853" height="480" src="https://www.youtube.com/embed/99TRpAmZFYM" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

I borrowed some code from Unity's [tutorial on platform games](https://unity3d.com/learn/tutorials/topics/2d-game-creation/creating-basic-platformer-game) to build a "Platform Controller" for Mr Nibbles. This is the thing that actually controls the how mr nibbles moves given an input.

I added a little of flourish in the way of big tubes to represent the entrance and exit points for the level. I even wrote some code that animated Mr Nibbles through the tubes but I found that it caused issues in the training so I decided to drop that aspect for now, perhaps ill add it again later.

The end result is that you are able to play a number of levels as mr nibbles. When you reach the exit another level is randomly selected and Mr Nibbles is spawned into that.

<iframe width="853" height="480" src="https://www.youtube.com/embed/IHdPzaLRY98" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

# The Agent

With the levels built its time to code up the agent.

## State

What information do we need to tell the network about the state of the world that will allow it to successfully learn how to navigate it?

Well we obviously need to supply the position of Mr Nibbles and the position of the exit point.

```csharp
public override List<float> CollectState()
{
    var state = new List<float>();

    state.Add(_exitPoint.transform.position.x);
    state.Add(_exitPoint.transform.position.y);

    state.Add(_platformController.transform.position.x);
    state.Add(_platformController.transform.position.y);

    return state;
}
```

We should probably also supply a variable to help with learning the potential set of actions depending if the player is in the air or not.

```csharp
public override List<float> CollectState()
{
    var state = new List<float>();

    ...

    state.Add(_player.IsGrounded ? 1 : 0);

    return state;
}
```

Im not sure if this is really needed but I thought it cant hurt.

Now for the hard part. How do I represent the level? I cant really pass in the collision bounds for each tile as that would be too complicated to represent as floats.

Each level is constructed as tiles, so everything sits on a uniform grid. So I should only need to pass in the centre point of each tile in the level along with its type and hopefully that should be enough for the agent from.

[![](./tile-xy-pos.jpg)](./tile-xy-pos.jpg)

The next issue is; how do I deal with different level sizes? [This GitHub issue](https://github.com/Unity-Technologies/ml-agents/issues/139) mentions that the state size needs to be fixed (I believe this is to do with the way TensorFlow works under the hood).

This restriction is going to make things a little tricky because our levels are different sizes with a different number of tiles, so keeping the state size the same for each level may be a problem (more on potential solutions to that later).

For now my "just get it working" solution is just to make sure the levels are quite small then grab all the tiles in a certain bounds. If the tile is empty I set its type to "0" if its a solid tile its "1" and if its a spider its "2".

[![](./tile-types-rendered.jpg)](./tile-types-rendered.jpg)

When the level changes I grab the tiles from the tilemaps:

```csharp
public BoundsInt tileBoundsToIncludeInState = new BoundsInt(-25, -1, 0, 40, 1, 1);

public override void AgentReset()
{
    ...
    _tiles = _game.CurrentLevel.GetComponentInChildren<TilesController>().GetTiles(tileBoundsToIncludeInState);
    ...
}
```

Then feed them into the state list:

```csharp
public override List<float> CollectState()
{
    ...

    foreach (var tile in _tiles)
    {
        state.Add(tile.position.x);
        state.Add(tile.position.y);
        state.Add(tile.type);
    }

    ...
}
```

## Actions

Now we have our state, we need to allow the agent to take and action and receive a reward or punishment for that action.

```csharp
public override void AgentStep(float[] actions)
{
    PerformActions(actions);
    UpdateRewards();
}
```

Digging into `PerformActions` first:

```csharp
public const int None = 0;
public const int MoveLeft = 1;
public const int MoveRight = 2;
public const int Jump = 3;

...

private void PerformActions(float[] actions)
{
    var nothing = (int) actions[None] == 1;
    var moveLeft = (int) actions[MoveLeft] == 1;
    var moveRight = (int) actions[MoveRight] == 1;
    var jump = (int) actions[Jump] == 1;

    var isJumping = false;
    var hozMove = 0f;

    if (moveLeft)
        hozMove = -1f;
    if (moveRight)
        hozMove = 1f;
    if (jump)
        isJumping = true;

    _platformController.Tick(hozMove, isJumping);
}
```

I handle the condition where the player can perform none, one or all of the actions simultaneously. I then pass those actions to the platform controller component which updates the physics of mr nibbles.

Now we need to handle how to reward the agent for its actions:

```csharp
private void UpdateRewards()
{
    if (_exitPoint.IsTriggered)
    {
        Wins++;
        reward = 10;
        done = true;
    }
    else if (_spiders.IsTriggered)
    {
        Deaths++;
        reward = -10;
        done = true;
    }
    else
    {
        reward = -0.01f;
    }
}
```

The agent is rewarded for reaching the exit, punished for touching the spiders and given a small punishment for each moment that goes by that it doesnt reach the exit.

Im not sure if these are the correct values, I played around with a few different values but im not sure if these are correct or not.

# Defining Brain and Academy

[As with my last post](/machine-learning/a-game-developer-learns-machine-learning-a-little-deeper/) I am going to leave the academy empty for now, I dont need its features as I dont have multiple agents in my scene.

The brain is setup like so:

[![](./brain-setup.png)](./brain-setup.png)

We have defined the number of states: tiles (x,y,type) + agent state (x,y,isOnGround) + exit point (x,y) = 1505

The number of actions: jump, left, right, nothing (stand still).

The action space is set to continuous because we want to allow the agent to perform multiple actions at once, jump and move right for example.

The state space is continuous because we arent dealing with integer positions for things in the level rather we are using floats.

# Training

With the environment constructed lets move onto training. First I need to get the hyperparameters right for the PPO algorithm. I re-read [the unity best practices doc](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/best-practices-ppo.md) and took a guess at the following hyperparams:

```python
### General parameters
max_steps = 5e5 # Set maximum number of steps to run environment.
run_path = "mrnibbles/run14-differentrewards" # The sub-directory name for model and summary statistics
load_model = False # Whether to load a saved model.
train_model = True # Whether to train the model.
summary_freq = 10000 # Frequency at which to save training statistics.
save_freq = 50000 # Frequency at which to save model.
env_name = "mrnibbles" # Name of the training environment file.
curriculum_file = None

### Algorithm-specific parameters for tuning
gamma = 0.99 # Reward discount rate.
lambd = 0.95 # Lambda parameter for GAE.
time_horizon = 1500 # How many steps to collect per agent before adding to buffer.
beta = 1e-3 # Strength of entropy regularization
num_epoch = 6 # Number of gradient descent steps per batch of experiences.
num_layers = 2 # Number of hidden layers between state/observation encoding and value/policy layers.
epsilon = 0.2 # Acceptable threshold around ratio of old and new policy probabilities.
buffer_size = 10000 # How large the experience buffer should be before gradient descent.
learning_rate = 3e-4 # Model learning rate.
hidden_units = 64 # Number of units in hidden layer.
batch_size = 2500 # How many experiences per gradient descent update step.
normalize = False
```

I kicked off training and fired up TensorBoard. After a few different attempts I felt like I had something that was training so I let it run for an hour or so:

[![](./tensorboard.png)](./tensorboard.png)

It looks okay, we can see that the cumulative reward increases sharply and levels off and the episode length decreases as the agent learns to play the level.

When we try our model out in the game however we see this:

<iframe width="853" height="480" src="https://www.youtube.com/embed/S21Rkob0XfM" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

On the positive side, it looks like he is attempting to move towards the exit point when the exit is to the right of the spawn point. On the negative side he doesnt ever seem to want to try going left and also seems to get stuck for a long time at a small hurdle.

I think whats going on is the levels where the exit point is to the left of the spawn point have spiders, thus they are significantly more difficult for the agent to solve than the levels where the exit point is to the right of the spawn point. As a result the agent has learnt that probably it should always just go right and is unable to learn that it can jump over the spiders.

It looks like the agent is having trouble with jumping in general..

# Future Work

I tried a few smaller tweaks but none seemed to have a significant effect. I think what I need to to change my training strategy.

## Curriculum Training

While browsing through the Unity docs I came across the concept of [curriculum training](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/curriculum.md). As I understand it; you incrementally train your agent with progressively more difficult scenarios thus accelerating the learning by focusing the agent on learning just one aspect of your game at a time. This reminds me of how humans learn in school, we aren't immediately thrown into quantum mechanics, instead we start with basic mathematics and work our way up from there.

I think my next set of experiments will have to take advantage of curriculum learning by slowly introducing level elements until the agent is able to successfully proceed.

## Tile Radius State

One current limitation of my current setup is that I encode the state of the entire level. This means I am currently limited to small levels of fixed size.

A solution (suggested by a member of [my local Machine Learning meetup group](https://perthmlgroup.wordpress.com/)) is to only provide the state of tiles that lie in a radius around Mr Nibbles. As Mr Nibbles moves so to does the nearest surrounding tiles. This is analogous to using a camera as my input state but unlike camera state the agent should be able to learn [temporal information](https://github.com/Unity-Technologies/ml-agents/issues/183).

## Reward Tweaking

I think I may need to work on how I reward and punish a little bit more. Perhaps I should have a timeout if it takes the agent too long to solve a level, maybe that would prevent it from learning that on certain levels its best just to suicide rather than try to solve the level.

## Velocity

Another potential issue is was unable to learn to jump correctly was because it was missing information about its velocity. I was hoping it could infer the velocity based on the previous positional state of mr nibbles but perhaps it would help if I manually supplied that information as two more state parameters.

# Conclusion

Well I cant really call this post a success, but I certainty learnt a lot and that's the ultimate goal. I have lots of ideas for how to proceed from here so subscribe to stay tuned.

Until next time!

P.S. If you want to see the code for this post then checkout this tag: https://github.com/mikecann/MrNibblesML/tree/MrNibblesBasicsBlogPost

Wow, I cant believe its been one whole year since [the last Govhack](https://www.mikecann.blog/portfolio/projects/govhack-2015-should-i-drive/), the hackathon where groups of people use government data to hack together a project over the course of a weekend.



I had a great time last year on the "[Should I Drive?](https://www.mikecann.blog/portfolio/projects/govhack-2015-should-i-drive)" team. We used WA Main Roads and other data sources to try to answer the question "should I drive to my destination or take some other form of transport?".

This year I set myself a goal. I wanted to do something to do with Machine Learning / AI. Its a field of computing thats really hot right now and im really interested in getting involved and learning more.

So as with [last year](https://www.mikecann.blog/portfolio/projects/govhack-2015-should-i-drive), after a brief welcome presentation by the organizers, competitors were invited to take the microphone and pitch ideas. Off-stage the pitchers then threw together a quick poster with their main ideas.

[![2016-07-29 18.52.00](https://www.mikecann.blog/wp-content/uploads/2016/08/2016-07-29-18.52.00-1024x768.jpg)](https://www.mikecann.blog/wp-content/uploads/2016/08/2016-07-29-18.52.00.jpg)

There were a coupple of interesting ones but the one that really caught my attention was from a guy who wanted to apply Machine Learning to old historic photographs. After a brief discussion with him around his poster I immediately signed up. 20 minutes later we had a team and were heading off upstairs to find a quiet area of the (awesome) [Flux co-working space](https://www.fluxperth.com/) where Govhack was being held this year.

In total we had 8 members, 3 technical and 5 non-technical. That first evening was mostly spent planning who was going to do what and what the priorities were. The three of us technical people sketched out our plan and divvied up the work so that we were all working efficiently.

[![3tech](https://www.mikecann.blog/wp-content/uploads/2016/08/3tech-1024x683.jpg)](https://www.mikecann.blog/wp-content/uploads/2016/08/3tech.jpg)

I took the front-end website and backend node host / API while Dominic took the Python code which would interface with the various data sets and Houraan did the code which would apply the Machine Learning to the images returned from the data sets.

[![theplan](https://www.mikecann.blog/wp-content/uploads/2016/08/theplan-1024x683.jpg)](https://www.mikecann.blog/wp-content/uploads/2016/08/theplan.jpg)

With the plan sorted I knew I had a bunch of work to get done before the non-technical people could get their hands on something to use and experiment with. With that in mind I decided to get up really early (4am) on Saturday morning and start work on the site. Im glad I did as it took me most of the morning to get the shell of an app up and running using ReactJS with Typescript and a NodeJS backend hosted on Heroku (such is the way with modern Javascript).

[![chrome_2016-08-01_12-54-14](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_12-54-14-1024x830.png)](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_12-54-14.png)

When I arrived at Flux later that morning, I demoed my progress and we discussed the scope of the project. The original idea was to produce videos from several photos in a sort of slideshow but after some discussion we decided to narrow the scope so that we were more likely to finish it in time. We decided that if we could just take old photos and apply ML to "Colourise" them then would be a cool way to explore the past using a modern technique.

With the scope of the project resolved, our next task was to come up with a name for it. One of our non-technicals Karl came up with "Colourful Past", we all agreed that it fit the scope and described the project perfectly.

The rest of the day was spent furiously hacking away on various facets of the project.

We setup a [Trello Board](https://trello.com/b/en3HeDLx/colourful-past) to manage tasks and a place to store links and other information about the project.

[![chrome_2016-08-01_13-04-34](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_13-04-34-1024x804.png)](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_13-04-34.png)

We used [Slack](https://slack.com/) for general communications and link sharing when we couldnt just shout across the table.

[![slack_2016-08-01_13-48-55](https://www.mikecann.blog/wp-content/uploads/2016/08/slack_2016-08-01_13-48-55-1024x718.png)](https://www.mikecann.blog/wp-content/uploads/2016/08/slack_2016-08-01_13-48-55.png)

Source code was uploaded to our github org [https://github.com/colourful-past](https://github.com/colourful-past):

[![chrome_2016-08-01_15-40-43](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_15-40-43.png)](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_15-40-43.png)

In general things went really smoothly. The technical side worked really well, we were able to work efficiently independently then combine the results towards the end. On the non-technical side there few a few issues managing tasks, keeping everyone working all the time but in general we were able to work effectively together.

By the end of the first day we had a basic but working product. You could type in a search term such as "Anzac Day", the client would then send an API request to the NodeJS server which would then in parallel call a number of Python scripts to query various datasets, the results were then aggregated and returned to the client.

[![chrome_2016-08-01_14-28-58](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_14-28-58.png)](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_14-28-58.png)

The user could then click a button to "Colourise" the black and white image. This makes another call to NodeJS which calls another Python script which uses a Machine Learning model developed by UC Berkeley and trained on 1.3 million black and white images to generate a coloured version of our historic photograph. The resulting image is stored in S3 and the URL returned to the client.

[![chrome_2016-08-01_14-35-48](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_14-35-48.png)](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_14-35-48.png)
(above image is just a placeholder and was not generated by the AI)

After I left on the Saturday evening, to get some needed sleep, Houraan soldiered on and gave the site some much needed design love. When I woke in the morning the site looked much improved, Houraan had done a phenomenal job.

[![chrome_2016-08-01_14-41-27](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_14-41-27.png)](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_14-41-27.png)

[![chrome_2016-08-01_14-42-59](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_14-42-59.png)](https://www.mikecann.blog/wp-content/uploads/2016/08/chrome_2016-08-01_14-42-59.png)

Sunday was our final day and we spent the first half adding the last bit of polish to the site, such as a really cool subtle gradient effect on the text:

[![2016-08-01_14-48-13](https://www.mikecann.blog/wp-content/uploads/2016/08/2016-08-01_14-48-13.gif)](https://www.mikecann.blog/wp-content/uploads/2016/08/2016-08-01_14-48-13.gif)

Then we concentrated on the presentation and video which the judges will be using later that day. Molly did a great job putting together our video which was uploaded to Youtube at the very last minute:

<iframe width="853" height="480" src="https://www.youtube.com/embed/kuNnUWMXwXs" frameborder="0" allowfullscreen></iframe>

The presentation worked a little different from last year. Instead of everyone getting up on a stage and doing a slideshow in front of the judges, the judges came round to each team's desk where we did a demo and a short talk before being asked a number of questions. Houraan, Karoline and Bruce nailed it for us while the rest of us watched and gave moral support.

[![2016-07-31 15.33.01](https://www.mikecann.blog/wp-content/uploads/2016/08/2016-07-31-15.33.01-1024x862.jpg)](https://www.mikecann.blog/wp-content/uploads/2016/08/2016-07-31-15.33.01.jpg)
(We had a little spare time so we thought we would play on the "Colourful" aspect of our product a little by blowing up some balloons, producing a poster and dressing in bright colours)

All in all it went really well and im very happy with the result. The judges seemed to think so too as we came away with a 1st prize in the "West Australian Community Prize" category.

[![2016-07-31 19.33.43](https://www.mikecann.blog/wp-content/uploads/2016/08/2016-07-31-19.33.43-1024x768.jpg)](https://www.mikecann.blog/wp-content/uploads/2016/08/2016-07-31-19.33.43.jpg)

Well thats just about it. If you want to have a look at what we produced you can try it out at [https://colourfulpast.org/](https://colourfulpast.org/). We dont know how long we are going to be able to keep the expensive AWS GPU instances going for so if you are viewing this post some time from now it might not work for you.

I just want to say a massive thanks to all my team-mates for making it an awesome weekend of fun hacking, thanks guys, I hope you see you all next year!

[![IMG_0389](https://www.mikecann.blog/wp-content/uploads/2016/08/IMG_0389-300x200.jpg)](https://www.mikecann.blog/wp-content/uploads/2016/08/IMG_0389.jpg)[![IMG_0455](https://www.mikecann.blog/wp-content/uploads/2016/08/IMG_0455-300x200.jpg)](https://www.mikecann.blog/wp-content/uploads/2016/08/IMG_0455.jpg)[![IMG_0436](https://www.mikecann.blog/wp-content/uploads/2016/08/IMG_0436-300x200.jpg)](https://www.mikecann.blog/wp-content/uploads/2016/08/IMG_0436.jpg)[![IMG_0395](https://www.mikecann.blog/wp-content/uploads/2016/08/IMG_0395-300x200.jpg)](https://www.mikecann.blog/wp-content/uploads/2016/08/IMG_0395.jpg)

(P.S. Big thanks to Karl for taking all the pictures!)