I cant code, but Im building stats tools for trading on my own with several AI agents on three machines. I dont let a agent approve its own work, so a second agent on another machine recompute the results using only the written method without looking at the first agents code.
Today that caught 2 numbers that only worked under assumption nobody had written down, an import that crashed above about 150K rows and an outdated threshold in the text. All the first agents tests had passed. The traces tell me what it did, but I still need another agent to check the numbers.
Unit tests? Or plain ordinary testing the same way you would do for other software? (e.g. even if it not software the same idea can apply)
I have a QR code redirect system that generates unique codes for business cards, flyers and such. Previously it generated PNG files but I wanted to load a PDF template, merge in N unique QR codes, and make a new PDF with N pages. I talked it over with Claude, who offered me a test plan although I didn't ask for it -- it was roughly the same test plan I would have made myself. Like I made some flyers, checked that they looked right, checked that the QR code scanned and redirected to the right place, etc.
Claude did a pretty good job, i had to do some visual tweaks, add something manually to the back end to accomodate it and make some little fixes. If I had a junior programmer do the task it would have been the same story.
Whatever you are trying to make by any means whatsoever you are going to have to convince yourself you like the result in the end.
I cant code, but Im building stats tools for trading on my own with several AI agents on three machines. I dont let a agent approve its own work, so a second agent on another machine recompute the results using only the written method without looking at the first agents code. Today that caught 2 numbers that only worked under assumption nobody had written down, an import that crashed above about 150K rows and an outdated threshold in the text. All the first agents tests had passed. The traces tell me what it did, but I still need another agent to check the numbers.
we built zeroshot.sh to let you mechanically enforce this flow (and any other topology), want to check it out? :)
Unit tests? Or plain ordinary testing the same way you would do for other software? (e.g. even if it not software the same idea can apply)
I have a QR code redirect system that generates unique codes for business cards, flyers and such. Previously it generated PNG files but I wanted to load a PDF template, merge in N unique QR codes, and make a new PDF with N pages. I talked it over with Claude, who offered me a test plan although I didn't ask for it -- it was roughly the same test plan I would have made myself. Like I made some flyers, checked that they looked right, checked that the QR code scanned and redirected to the right place, etc.
Claude did a pretty good job, i had to do some visual tweaks, add something manually to the back end to accomodate it and make some little fixes. If I had a junior programmer do the task it would have been the same story.
Whatever you are trying to make by any means whatsoever you are going to have to convince yourself you like the result in the end.