You’ll have seen the adverts on TV. Well, I did too. And I thought – have they possibly changed in the few decades I’ve known them? They’ve always been in the back of mind – but not in a good way (especially when it came to domains). Has the rebranding done any good?
I like a bargain as much as anybody else does, and although I’ve been very happy with DigitalOcean, 1&1 IONOS’ VPS service for £1.20/month for 6 months before another 6 months of £24/month seemed quite reasonable for the specifications on offer (4 vCPU, 8Gb RAM and 160Gb SSD).
I know my own address, thanks..
So I signed up early last week. The first thing that drove me insane was their postcode/address lookup function when entering your address as a new customer. I have constant problems with postcode databases not getting my address properly and 1&1 are no different. After entering my postcode, the system told me my address was wrong and I couldn’t move forward with completing the registration form unless I accepted their version of my address (which is wrong). So I just accepted it. When it came to payment, a similar problem, but the system seemed to accept it and was charged £1.20 just fine.
It wasn’t until later the following day I received the account set-up confirmation email and I proceeded to log in and start getting things set-up. The very first to do was to lock down the server so that only I could connect to it from my home and work IP addresses for the purposes of SSH access (command line access). 1&1 IONOS comes with a firewall, so I started to configure it. As I also use CloudFlare for caching, WAF and firewall, I started to configure the IONOS firewall for that – though I note that the documentation for the firewall doesn’t mention you can use CIDR notation for the allowed IPs. The web form will accept them though! According to the official firewall docs, you can specify a range of IPs with a dash, but since CIDR is a perfectly normal and standard notation for IP ranges, I’d try that (it saves typing). After a while (as CloudFlare has a fair number of IP ranges), everything looked set to go. CloudFlare’s servers were the only ones that could connect to TCP port 443.
Let me explain how CloudFlare works, as you’ll find that neither 1&1 IONOS engineers or my “personal consultant” understand how systems like CloudFlare or Akamai work (I’ve been using CloudFlare for at least 7 or 8 years, and Akamai for 2).
How does CloudFlare work?
When you request a page from my blog, the request goes to CloudFlare. CloudFlare does a few security checks first of all, then, if you’re not a naughty bot or person, it checks to see if the page already exists in its cache. If not, CloudFlare – and ONLY CloudFlare – will connect to my VPS securely to retrieve the page and serve it to you. You, as a requester cannot bypass CloudFlare to get to my VPS directly unless I specifically disable proxying within CloudFlare (my DNS is hosted with CloudFlare so any changes I make should be almost immediate).
Too hot to trot?
When I set-up the 1&1 IONOS VPS server, it took me about 30 minutes to get everything running including moving everything off DigitalOcean and installing MySQL, PHP and nginx. I’ve written scripts which perform much of the set-up for me – and everything is checked into BitBucket so that I can retrieve those scripts at any time from anywhere. I also have many backups at Backblaze B2, courtesy of rclone (written and maintained by my former boss at Memset Hosting Ltd.)
CloudFlare could not talk to the 1&1 IONOS VPS. Connection timed out every time. I set-up a firewall rule to allow myself direct access to the VPS via port 443) to test that the LNMP stack was working correctly. It was. Output from netstat showed everything was fine. No local firewall was running, and iptables rules were clear and set to accept. And yes, I had changed the IP addresses in CloudFlare’ DNS to the new shiny VPS.
How about you try turning it on and off again?
So I utilised 1&1 IONOS’ live chat system for technical support. They’re fast, but they wanted to know why I was locking off port 443 to specific IPs. I explained I was using CloudFlare. I checked with them if the syntax of the firewall rules were correct. Apparently, they were. Their advice? Open port 443 to the world. I asked them if they had any experience with CloudFlare or Akamai or any other similar service. The whole point with these systems is that it acts as a barrier between the internet at large and your origin servers. The origins which host your application should never be exposed externally but only through CloudFlare, Akamai or whoever.
So I called my “personal consultant” for help by submitting a request for a callback. Within a minute or two I was connected. I explained the problem to him and he went away and spoke to the technical people. Their explanation was how CloudFlare was returning client IPs. Which is absolute bull. See my explanation further above. The connecting IPs are the ones that I defined in the firewall. The same IPs I had been using at DigitalOcean with their firewall. Client IPs that come in are passed to the original server in the form of a header (and my nginx configuration looks at that header and parses the real IP which is then available in the server logs) – but that’s got nothing whatsoever to do with CloudFlare’s servers connecting to my VPS.
No experienced sysadmin should touch 1&1 with a bargepole..
While I was still talking to the chap on the phone, the blog suddenly spluttered into life. But it is not obvious why. The firewall rules didn’t look to have changed. But still, I didn’t like the explanation whatsoever from their technical department as to how CloudFlare operates and the encouragement of opening TCP port 443 to the world. I had to explain that I’m a systems administrator of some 22 years, having worked for two Academy Award-winning VFX companies, and now help manage multi-million-pound websites for some very high profile clients and have extensive experience with CloudFlare. So I cancelled the account there and then.
I was transferred to the US division of 1&1 for cancellation. But after 10 minutes or so, I was put through to the right person who cancelled the account for me. And I received this email:
I’ve gone back to DigitalOcean again (~£10/month for third of the resources). But in order to test my DR (disaster recovery) plan, wiped the old server, set-up a new one, and restored everything from my Backblaze B2 backups. It all works perfectly.
CloudFlare had no problem connecting to my new VPS at DigitalOcean. New IP and everything. That’s how 1&1 IONOS should have worked out of the box. I blame their firewall and their documentation. And possibly lack of experience of IDS/WAF/CDN systems such as CloudFlare.