Bought a bunch for a client project. Pretty decent form-factor, but they really need to have hardware that can do on-device AI inference. There are lots of reasonably priced choices out there.
Their whole “private” voice assistant story falls apart when they have to send everything to the cloud, including third-party services.
Most appliances connect using a cheap microprocessor like an ESP-32 which provides both WiFi and BLE.
If there’s a DNS server running, it’s usually for mDNS/ZeroConf service discovery. Usually so it can easily get provisioned via a smart speaker or hub.
More details here: https://circuitlabs.net/dns-server-and-dns-sd-implementation/
Have done it both ways. Will never go back to bare metal. Dependency hell forced multiple clean installs down to bootloader.
The only constant is change.
There goes any chance Intel had at competing in data-center AI processors.
Sounds like the issue is getting to the server, not the LLM server itself. If so, may want to look into running a reverse proxy, or if you want to access it remotely, tunnels: https://github.com/anderspitman/awesome-tunneling
AWS S3 lets you upload all content to a bucket, then mark it as a website. If usage is not too heavy, it can stay under the free tier.
But a favorite free one is Cloudflare pages: https://www.geeksforgeeks.org/techtips/deploying-static-website-to-cloudflare-pages/
You can keep your content on github, connect it to a CF page, and have it auto-update on push to github.
How do you go from 3D game hosting for kids to… dating?
I am the happiest person ever. Life to me was never about accomplishment, but about Happiness, which is Smiles minus Frowns.
Words to live by.
A few suggestions:
Some of those components may end up costing a lot to operate. You said you’re doing it as a portfolio piece. May want to create a spreadsheet with all the services, then run a cost simulation. You can use the AWS Cost calculator, but it won’t be as flexible doing ‘what if’ scenarios. Any prospective employer will appreciate that you’ve given some thought to runtime pricing.
You may want to bifurcate static media out and put them in S3 buckets, plus put a CloudFront CDN in front for regional scaling (and cost). Static media coming out of local server uses up processing power, bandwidth, storage, and memory. S3/CloudFront is designed for just this and is a lot cheaper. All fonts, js scripts, images, CSS stylesheets, videos, etc. can be moved out.
Definitely expire your CloudWatch log records (maybe no more than a week), otherwise they’ll pile up and end up costing a lot.
Consider where backups and logs may go. Backups should also account for Disaster Recovery (DR). Is the purpose of multiple AZs for scaling or DR? If for DR, you should think about different recovery strategies and how much down-time is acceptable.
Using Pulumi is good if the goal is to go multi-cloud. But if you’ve hardcoded Aurora or ALBs into the stack, you’re stuck with AWS. If that’s the case, maybe consider going with AWS CDK in a language you like. It would get you farther and let you do more native DevOps.
Consider how updates and revisions might work, especially once rolled out. What scripts will you need to run to upgrade the NextCloud stack. What are the implications if only one AZ is updated, but not the other. Etc.
If this is meant for business or multiple users, consider where user accounts would go? What about OAuth or 2FA? If it’s a business, they may already have an Identity Provider (IDP) and now you need to tie into it.
If tire-kicking, may want to also script switching to plain old RDS/Postgres so you can stay under the free tier.
To make this all reusable, you want to take whatever is generated (i.e. Aurora endpoints, and save everything to a JSON or .env file. This way, the whole thing can be zapped and re-created and should work without having to manually do much in the console or CLI.
Any step that uses the console or CLI adds friction and risk. Either automate them, or document the crap out of them as a favor to your future self.
All secrets could go in .env files (which should be in .gitignore). Aurora/RDS Database passwords could also be auto-generated and kept in SecretsManager and periodically rotated. Hardcoded DB passwords are a risk.
Think about putting WAF in front of everything with web access to prevent DDOS attacks.
This is a great, learning exercise. Hope you don’t find these suggestions overwhelming. They only apply if you want to show it off for future employers. If it’s just for personal use, ignore all the rest I said and just think about operating costs. See if you can find an AWS sales or support person and get some freebie credits.
Best of luck!
In alarm away mode, if motion sensor in the middle of the house is triggered without having set off the motion sensors elsewhere, disable that sensor until alarm disabled.
Friggin’ hyperactive cat, doing leaping zoomies.
In my old consulting days I got a lot of referrals from startups who had gone down the rabbithole making ‘cross-platform’ mobile apps. They had all gone as far as they could, then hit a brick wall with performance or lack of technical support. The solution always came down to taking the hit and rewriting it properly, or trudging onward.
Getting a lot of the same vibe with this ‘casual AI coder’ hype. Best of luck.
The one guy hand-soldering and fumes with no PPE or vent 😱
If nginx, here’s an open-source blocker/honeypot: https://github.com/raminf/RoboNope-nginx
If you have it set up to be proxied or hosted by Cloudflare, they have their own solution: https://blog.cloudflare.com/declaring-your-aindependence-block-ai-bots-scrapers-and-crawlers-with-a-single-click/
12K. brought to you by Hollywood Face Makeup and CGI alliance.
I’ve had good luck with Jekyll, saving the source on github, and setting it up so pushing to main auto-deploys to a Cloudflare site. Using Markdown and for larger media, uploading to S3.
Much easier to set up and maintain than github pages. Since it’s static output, pretty snappy. Also includes RSS feeds and permanent URLs.
Have also set up several Wordpress sites. Slower, but if you want wysiwyg editing, user comments, or there are several contributors, would work better.
Have also heard good things about ghost, but haven’t actually deployed one yet.
You’re overpaying 😁
Cloudflare static web hosting, including TLS/SSL, DDOS protection, WAF, and AI scraper protection, are all free: https://softwareonbudget.com/blog/how-to-host-static-website-for-free-with-cloudflare-pages/
And if you connect it to github repo, it auto-updates on push to main.
No connection. Just a happy user and a fan.
Saw a posting this past week on SSD drive failures. They’re blaming a lot of it on ‘over-logging’ – too much writing trivial, unnecessary data to logs. I imagine it gets worse when realtime data like OpenTelemetry get involved.
Until I saw that, never thought there was such a thing as ‘too much logging.’ Wonder if there are any ways around it, other than putting logs on spinny disks.
I did set up a local LLM on a pretty beefy machine and Whisper to do local voice assistance, but it kept falling apart. The only one that worked reasonably was tying it to a commercial API. This was more than a year ago, so things might have improved.
But if they want to sell these things as an Alexa/Google Home alternative, they’ll have a hell of a story if they built a one-stop hub that ran an on-device model and did everything locally. There are a lot of smaller models now and companies like MediaTek make decent edge-processors with beefy NPUs.
I’m still a big fan of their approach. Sad to see them drop the yellow, but I hope it means they’ll come out with something better.