I actually wanted to automate some large scale testing for vintage story. Figured i would set my local LLM on the task to sort it out just to see what it would do. I drafted up a nearly three page document with clear instructions, rules, tools, examples and goals. Put hard limits on the sandbox the LLM runs in so that it couldn’t choose to just ignore the rules that could cause security issues and i let it lose.
It started with basic mouse and keyboard inputs and figured out by it self how to launch the game and run it though the user interface. After about 4 hours it stopped. Stated in its logic that what it was doing is “inefficient and wasting time” Then proceeded to promptly start working on a way to directly interface with it by designing a bot, getting a smaller model i had on file that could load along side it and drive the bot. It then started working on the hard problems would hand basic instructions to the smaller llm and it would drive a bot that loaded into the game as a mod.
After about 12 hours of total work it basically created a useful and well designed and functional vintage story bot and testing system. Would have likely taken me twice as long to design the bot.
Its been working well for about two weeks now. If i had just vibed out a half assed request or put in no hard safeguards outside of the LLMs control it likely would have done something fucking stupid. As with anything, its almost ALWAYS user error. And only an idiot blames their tools for their own fault.
I actually wanted to automate some large scale testing for vintage story. Figured i would set my local LLM on the task to sort it out just to see what it would do. I drafted up a nearly three page document with clear instructions, rules, tools, examples and goals. Put hard limits on the sandbox the LLM runs in so that it couldn’t choose to just ignore the rules that could cause security issues and i let it lose.
It started with basic mouse and keyboard inputs and figured out by it self how to launch the game and run it though the user interface. After about 4 hours it stopped. Stated in its logic that what it was doing is “inefficient and wasting time” Then proceeded to promptly start working on a way to directly interface with it by designing a bot, getting a smaller model i had on file that could load along side it and drive the bot. It then started working on the hard problems would hand basic instructions to the smaller llm and it would drive a bot that loaded into the game as a mod.
After about 12 hours of total work it basically created a useful and well designed and functional vintage story bot and testing system. Would have likely taken me twice as long to design the bot.
Its been working well for about two weeks now. If i had just vibed out a half assed request or put in no hard safeguards outside of the LLMs control it likely would have done something fucking stupid. As with anything, its almost ALWAYS user error. And only an idiot blames their tools for their own fault.