"Vibe coding" isn't a "skill", is a meme or a experiment, something you do for fun, not for writing serious code where you have a stake in the results.
Programming together with AI however, is a skill, mostly based on how well you can communicate (with machines or other humans) and how well your high-level software engineering skills are. You need to learn what it can and cannot do, before you can be effective with it.
Call it "AI programming" or "AI pairing" or "Pair programming with AI" or whatever else, "vibe coding" was "coined" with the explicit meaning of "I'm going by the vibes, I don't even look at the code". If "vibe coding" suddenly mean "LLM was involved somehow", then what is the "vibe" even for anymore?
I agree there is a spectrum, and all the way to the left you have "vibe coding" and all the way to the right you have "manual programming without AI", of course it's fine to be somewhere in the middle, but you're not doing "vibe coding" in the way Karpathy first meant it.
Happened to me a few weeks ago. I replied/filled out the form, and after a day it was unlocked. Seems to be very hit and miss, maybe depending on who is seeing your replies? Regardless, definitely a sucky issue...
There are different types of attacks possible though, most broadly you can divide them into "design holes" and "implementation holes". This seems to be about preventing a design hole, and those you need to prevent with architecture/design, you can't just fix those once the implementation and documentation is done.
But the design hole is treating (IMO unfortunately) transient identities (the web's domain name system) as something that should persistently identify something. Adding hacks like this doesn't fix the underlying mismatch but creates new issues as seen in the article.
Or to put it another way: Domain names changing hands is how the web works. If you design your system to support web identities in a way that domains can't change hands then you are not supporting web identities but rather something different.
I totally agree. This is the fundamental reason I've stayed off there until now. I'm not about to tie my permanent identity to a domain which I might not own tomorrow. And did:plc is not an option for obvious reasons.
Even though naively I treat my own domain as my online identity I see what you mean since they can be taken away by actions outside of our control.
What would work better though? Like are we talking a more hardened identification system tied to personal data that can't change? If that's the case are there negative privacy effects of that, especially with whatever system controls that data?
Probably about the same, there is a lot of VPS providers out there, and not a small
amount accepts basically an email + cryptocurrencies without any further verification than that. And that's just on the clearweb, going beyond that you start having even more options.
Yeah, although the smaller providers, the sketchier they are. I'd rather use a VPN in a pool of thousands/millions of users. As a data point, I can signup for Proton VPN by downloading it to my iPhone and providing any email address. Without any payment, I can connect to VPN servers and browse anonymously ("anonymously"). This is certainly easier than provisioning a new VPS, not least because I need to pay for it.
> by politicians who think the internet is something they can aggressively control
You seem to believe they're wrong. Since they're the ones who come up with the laws of the land, I think it's important to realize that they can and do aggressively control access to the internet in their country. It sucks, but it's the reality.
> they can and do aggressively control access to the internet
yes but this is like watching someone deal with an ant infestation by stamping on them. They're not solving the issue and unlike the ant analogy, they're making the problem worse.
I've been met with that kind of stone walling before too, you know what eventually worked to actually turn the position of a local councilwoman? Going to her office and demand to speak with her, then sitting down, listening and having a conversation with her. Turns out that most of the emails "she" wrote to me was written by an assistant "to save her time" and she weren't aware of the points I was trying to bring up. Granted, this was like one and half decade ago, but if I was met with something similar today I'd try the same thing.
People tend to be a lot more reasonable in person, and also if you listen to them first.
Yeah, also they could be male. Don't take it so literal, the point I'm making is about going and physically meeting people, not about what title/label those people have.
> decide if the code it wrote or the tests it wrote are wrong
Personally I think it's too early for this. Either you need to strictly control the code, or you need to strictly control the tests, if you let AI do both, it'll take shortcuts and misunderstandings will much easier propagate and solidify.
Personally I chose to tightly control the tests, as most tests LLMs tend to create are utter shit, and it's very obvious. You can prompt against this, but eventually they find a hole in your reasoning and figure out a way of making the tests pass while not actually exercising the code it should exercise with the tests.
I haven’t found that to be the case in practice. There is a limit on how big the code can be so it can do it like this, and it still can’t reliably subdivide problems on its own (yet?), but give it a module that is small enough it can write the code and the tests for it.
You should never let the LLM look at code when writing tests, so you need to have it figure out the interface ahead of time. Ideally, you wouldn’t let it look at tests when it was writing code, but it needs to tell which one was wrong. I haven’t been able to add an investigator into my workflow yet, so I’m just letting the code writer run and evaluate test correctness (but adding an investigator to do this instead would avoid confirmation bias, what you call it finding a loophole).
> I haven’t found that to be the case in practice.
Do you have any public test code you could share? Or create even, should be fast.
I'm asking because I hear this constantly from people, and since most people don't have as high standards for their testing code as the rest of the code, it tends to be a half-truth, and when you actually take a look at the tests, they're as messy and incorrect as you (I?) think.
I'd love to be proven wrong though, because writing good tests is hard, which currently I'm doing that part myself and not letting LLMs come up with the tests by itself.
I'm doing all my work at Google, so its not like I can share it so easily. Also, since GeminiCLI doesn't support sub-agents yet...I've had to get creative with how I implement my pipelines. The biggest challenge I've found ATM is controlling conversation context so you can control what the AI is looking at when you do things (e.g. not looking at code when writing tests!). I hope I can release what I'm doing eventually, although it isn't a key piece of AI tech (just a way to orchestrate the pipeline to make sure that AI gets different context for different parts of the pipeline steps, it might be obsolete after we get better support for orchestrating dev work in GeminiCLI or other dev-oriented AI front ends).
The tests can definitely be incorrect, and are often incorrect. You have to tell the AI that consider that the tests might be wrong, not the implementation, and it will generally take a closer look at things. They don't have to be "good" tests, just good enough tests to get the AI writing not crap code. Think very small unit tests that you normally wouldn't think about writing yourself.
> They don't have to be "good" tests, just good enough tests to get the AI writing not crap code. Think very small unit tests that you normally wouldn't think about writing yourself.
Yeah, those for me are all not "good tests", you don't want them in your codebase if you're aiming for a long-term project. Every single test has to make sense and be needed to confirm something, and should give clear signals when they fail, otherwise you end locking your entire codebase to things, because knowing what tests are actually needed or not becomes a mess.
Writing the tests and let the AI write the implementation ends you up with code you know what it does, and can confidently say what works vs not. When the IA ends up writing the tests, you often don't actually know what works or not, not even by scanning the test titles you often don't learn anything useful. How is one supposed to be able to guarantee any sort of quality like that?
If it clarifies anything, I have my workflow (each step is a separate prompt without preserved conversation context):
1 Create a test plan for N tests from the description. Note that this step doesn't provide specific data or logic for the test, it just plans out vaguely N tests that don't overlap too much.
2 Create an interface from the description
3 Create an implementation strategy from the description
4.N Create N tests, one at a time, from the test plan + interface (make sure the tests compile) (note each test is created in its own prompt without conversation context)
5 Create code using interface + implementation strategy + general knowledge, using N tests to validate it. Give feedback to 4.I if test I fails and AI decides it is the test's fault.
If anything changes in the description, the test plan is fixed, the tests are fixed, and that just propagates up to the code. You don't look at the tests unless you reach a situation where the AI can't fix the code or the tests (and you really need to help out).
This isn't really your quality pass, it is crap filter pass (the code should work in the sense that a programmer wrote something that they thinks works, but you can't really call it "tested" yet). Maybe you think I was claiming that this is all the testing that you'll need? No, you still need real tests as well as these small tests...
Testing is fun, but getting all the scaffolding in place to get to the fun part and do any testing suuuuucks. So let the LLM write the annoying parts (mocks. so many mocks.) while you do the fun part
reply