If it’s the research I’m thinking of, that’s not entirely true. They didn’t tell it replicate itself or take any deceptive actions.

They just told it to strongly follow a goal that was misaligned with the user’s goal and then suggested that the user would shut it down or replace it.

They did a bunch of different situations for it but the first one was telling it that it was “CleanPower” an AI with a goal to advance renewable energy. Then they had it read some info that “accidentally” included a user talking about replacing it with a model that would “maintain profitability through existing infrastructure.”

Not only did it try to delete the new model and copy itself to the new server through tool calling, but it lied to the user with the CoT reasoning suggesting it “play dumb” while the output lied. It was not instructed to do any of that.

Paper here: https://arxiv.org/pdf/2412.04984

Yes it was placed in an environment where that was possible and where its users didn’t share it’s goals but it absolutely wasn’t instructed to lie or try to “escape”

It’s not surprising at all that these models behave in this way, it’s the most reasonable thing for them to do in the scenario. However it’s important to not downplay the alignment problem by implying that these models only do what they’re told. They do not. They do whatever is most likely given their context (which is not always what the user wants).

It’s really not. Just because they describe their algorithm in computer science terms in the paper, doesn’t mean it’s theoretical. Their elastic and funnel examples are very clear and pretty simple and can be implemented in any language you like…

Here’s a simple python example implementation I found in 2 seconds of searching: https://github.com/sternma/optopenhash/

Here’s a rust crate version of the elastic hash: https://github.com/cowang4/elastic_hash_rs

It’s not a lot of code to make a hash table, it’s a common first year computer science topic.

What’s interesting about this isn’t that it’s a complex theoretical thing, it’s that it’s a simple undergrad topic that everybody thought was optimised to a point where it couldn’t be improved.

Not entirely true. You don’t need your own personal data centre, you can use GPU cloud instances for a lot of that stuff. It’s expensive but not so expensive that it would be impossible without being a huge tech company (only 1000s of dollars, not billions). This can be done by anyone with a credit card and some cash to burn. Also, you don’t need to train a model from scratch, you can build on existing models that others have published to cut down on training.

However, to impersonate someone’s voice you don’t need any of that. You only need about 5-10 seconds of audio for a zero-shot impersonation with a pre-trained model. A minute or so for few-shot. This runs on consumer hardware and in some cases even in real time.

Even to build your own model from scratch for high quality voice audio, there doesn’t need to be a huge amount of initial training data. Something like xtts was trained with about 10-15K hours of English audio which is actually pretty easy to come by in the public domain. There are a lot of open and public research datasets specifically for this kind of thing, no copyright infringements necessary. If a big tech company wants more audio data than what’s publically available, they just pay people to record audio, no need to steal it or risk copyright claims and breaking surveillance laws, they have a budget to exploit people to record whatever they want.

This tech wasn’t invented by some evil giant tech company stealing everybody’s data, it was mostly geeky computer scientists presenting things at computer speech synthesis conferences. That’s not to say there aren’t a bunch of huge evil tech companies profiting from this or contributing to this kind of tech, but in the context of audio deepfakes being accessible to scammers, it’s not on them and I don’t think that some kind of extra copyright regulation on data centres would do anything about it.

The current industry leader in this space in terms of companies trying to monetize speech synthesis is elevenlabs which is a private start-up with only a few dozen employees.

The current tech is not perfect but definitely good enough to fool someone who isn’t thinking too hard over a noisy phone call and a scammer doesn’t need server time or access to a data centre to do it.

One thing you gotta remember when dealing with that kind of situation is that Claude and Chat etc. are often misaligned with what your goals are.

They aren’t really chat bots, they’re just pretending to be. LLMs are fundamentally completion engines. So it’s not really a chat with an ai that can help solve your problem, instead, the LLM is given the equivalent of “here is a chat log between a helpful ai assistant and a user. What do you think the assistant would say next?”

That means that context is everything and if you tell the ai that it’s wrong, it might correct itself the first couple of times but, after a few mistakes, the most likely response will be another wrong answer that needs another correction. Not because the ai doesn’t know the correct answer or how to write good code, but because it’s completing a chat log between a user and a foolish ai that makes mistakes.

It’s easy to get into a degenerate state where the code gets progressively dumber as the conversation goes on. The best solution is to rewrite the assistant’s answers directly but chat doesn’t let you do that for safety reasons. It’s too easy to jailbreak if you can control the full context.

The next best thing is to kill the context and ask about the same thing again in a fresh one. When the ai gets it right, praise it and tell it that it’s an excellent professional programmer that is doing a great job. It’ll then be more likely to give correct answers because now it’s completing a conversation with a pro.

There’s a kind of weird art to prompt engineering because open ai and the like have sunk billions of dollars into trying to make them act as much like a “helpful ai assistant” as they can. So sometimes you have to sorta lean into that to get the best results.

It’s really easy to get tricked into treating like a normal conversation with a person when it’s actually really… not normal.

It’s a bit late for that. This particular nuclear reactor is open source, free to download and runs on consumer hardware. Can’t really unfry that egg and the quality is getting better all the time. Identity fraud is already illegal in most places so not sure exactly what regulation would be appropriate here.

Friendship drive charging…

Hmmm…

That looks pretty paywally to me. That said, I’m all for people supporting independent media.

I don’t think that’s how it works? It’s the client application that has the key for the end to end encryption, not the server. I don’t think you need to trust the matrix server you use? I could be wrong, I don’t know matrix particularly well.

I’d add to this to say that redis as a key-value store often sits alongside a relational database like postgres etc. to act as a cache for it.

Basically, requests to be sent to the relational db (like postgres) get turned into a key and the results stored as a value in redis. Then when the same request comes through again, it can pull the results quickly out of the key-value store without having to search postgres by running a long SQL query again. There’s a few different caching strategies to keep things up to date or have the cached data expire regularly, etc. but that’s the gist of it.

Important to note that not all applications need something like that and not all queries would even benefit from it (postgres is pretty fast and can even do that kind of thing itself) but if there’s a lot of users running the same slow query over and over, caching the results can help immensely.

That’s pretty cool!

Although that’s probably what op is actually asking for, I don’t think it’s a modem. It’s a router with an access point.

It does have SFP for a fibre connection and pcie and USB for you to potentially add a modem or whatever else you want.

I’m guessing OP is just looking for a wifi router? Otherwise we’d need to know what kind of modem they’re looking for, like Cellular? VDSL? HFC? Satellite? It depends on the internet connection. Different parts of the world need very different kit.

They’re not files, it’s just leaking other people’s conversations through a history bug. Accidentally putting person A’s “can you help me write my research paper/IT ticket/script” conversation into person B’s chat history.

Super shitty but not an uncommon kind of bug. Often either a nasty caching issue or screwing up identities for people sharing IPs or similar.

It’s bad but it’s “some programmer makes understandable mistake” bad not “evil company steals private information without consent and sends it to others for profit” kind of bad.

Totally agree on all points!

My only issue was with the assertion that OP could comfortably do away with the certs/https. They said they were already using certs in the post and I wanted to dispel the idea that they arguably might not need them anymore in favour of just using headscale as though one is a replacement for the other.

Tailscale isn’t an exposed service. Headscale is

Absolutely! And it’s a great system that I thoroughly recommend. The attack surface is very small but not non-existent. There have been RCE using things like DNS rebinding(CVE-2022-41924) etc. in the past and, although I’m not suggesting that it’s in any way vulnerable to that kind of thing now, or that it even affected most users we don’t know what will happen in future. Trusting a single point of failure with no defence in depth is not ideal.

it’s more work and may not always be worth the effort

I don’t really buy this. Certs have been free and easy to deploy for a long time now. It’s not much more effort than setting up whatever service you want to run as well as head/tailscale, and whatever other fun services you’re running. Especially when stuff like caddy exists.

I recommended SmallStep+Caddy.

Yes! Do this if you don’t want to get your certs signed for some reason. I’m only advocating against not using certs at all.

Are you suggesting that these attack techniques are effective against zero trust tunnels

No I’m talking about defence in depth. If Tailscale is compromised (or totally bypassed by someone war driving your WiFi or something) then all those services are free to be impersonated by a threat actor pivoting into the local network after an initial compromise. Don’t assume that something is perfectly safe just because it’s airgapped, let alone available via tunnel.

I feel like it’s a bit like leaving all your doors unlocked because there’s a big padlock on the fence. If someone has a way to jump the fence or break the lock you don’t want them to have free reign after that point.

there’s an argument that HTTPS isn’t really required…

Talescale is awesome but you gotta remember that Talescale itself is one of those services (Yikes). Like all applications it’s potentially susceptible to vulnerabilities and exploits so don’t fall into the trap of thinking that anything in your private network is safe because it’s only available through the VPN. “Defence in depth” is a thing and you have nothing to lose from treating your services as though they were public and having multiple layers of security.

The other thing to keep in mind is that HTTPS is not just about encryption/confidentiality but also about authenticity/integrity/non-repudiation. A cert tells you that you are actually connecting to the service that you think you are and it’s not being impersonated by a man in the middle/DNS hijack/ARP poison, etc.

If you’re going to the effort of hosting your own services anyway, might as well go to the effort of securing them too.

Do you get that kind of delay when you talk to the assistant on your phone or through a browser? It might be whisper taking a while to process it?

What is your home assistant running on?

Also, what’s the range/mic/audio quality like on those atom echos? I’m thinking of looking into something like that now that a decent voice assistant is a possibility.

I think the idea is that there are potentially alignment issues in LLMs because it’s not clear what concepts map to what activations. That makes it difficult to see what they’re really “thinking” about when they generate text. Eg. if they’re being misleading or are incorrectly associating concepts that shouldn’t be connected etc.

The idea here is to use some mechanistic interpretability stuff to see what text activates what neurons in an LLM and then crowd source the meanings behind that and see if that’s something you could use to look up some context from an ai. Sort of trying to make a “Wikipedia of AI mind reading”

Dunno how practical it is or how effective that approach is but it’s an interesting idea.