I picked up a link to a free online course from a recent Linux Voice podcast. I tried the course and wasn’t really impressed with it. The interface is nice, with an embedded virtual terminal to let you practice typing in the commands but the pedagogy is pretty weak. There’s nothing in the presentation that indicates that the designers understand how to construct understanding: all this lovely bit of code is doing is rehearsing a list of commands (and there’s even a click-once shortcut if you can’t be bothered actually typing). It’s more of a checklist than a course. It would be easy to turn it into a really effective bit of online learning with the addition of some better structure and graphics, and maybe a little assessment for learning. A shame, really, as it is clearly a loss leader to sell the Code School itself: I am in the market for some good quality online learning in their area, but I’m not likely to look any further at their catalogue.
For some time, I’ve wanted to have a calendaring tool independent of Google Calendar, which has become a central tool to my productivity and a source of concern as to how much data profiling results from it.
This afternoon, I installed the open source ownCloud file storage, calendar and contacts suite on my Centos VPS. It was a straightforward exercise:
- Create a subdomain on the server and switch it to use PHP 5.6. Add /dev/urandom to open_basedir in php settings.
- Make a data folder behind the web root, chowned to the web user.
- Create a MySQL database for the ownCloud service.
- In the web root folder, get the software:
curl -O https://download.owncloud.org/community/owncloud-10.0.3.tar.bz2
- Check the MD5 hash, chown and extract. Copy the extracted files into the root folder (be careful to include dotfiles, e.g. cp owncloud/* . and cp owncloud/.* .)
- Visit the domain to configure the installation.
What this server now provides is an independent calendar service, contacts, and secure file storage, at no additional cost and under my own secure control.
For those of us who make use of the amazing
get_iplayer program to obtain clips and other resources for classroom and other conveniences, it comes as a bit of a blow to find that in the past week or two, it has stopped working. Fortunately, there is a new version of the program available that with a little effort, gets the facility working again.
From the release notes:
The BBC removed all the XML-based data sources used by get_iplayer on 2017-04-26, breaking a lot of get_iplayer functionality. That functionality has been restored, but there are changes to be aware of – get_iplayer has not survived unscathed.
Phil Lewis and the team have (once again) done a fantastic job of quickly responding to changes in the way the BBC delivers its content. Many, many thanks to all the devs and hacks involved in this release.
Finally, my advice to users is to read the release notes carefully. You may also hit issues installing the new dependencies including Mojolicious and Perl as well as the cpan perl repository. Persevere, there is lots of useful advice out there. Finally, finally, the cache updates are much slower than before, although they are now only updated weekly.
Over fifty years ago, my father was a US Air Force signals operator: he, like any other professional in communication, had to learn the languages of communication, command and control. I still have the LP (long-playing record, what the kids call “vinyl” now, although these weren’t vinyl) record set that he listened to as he learned Morse Code.
Today’s young people live in a world of communication and it is increasingly important for them – and all users – to at least have an appreciation of the languages used by the systems that pervade our modern lives. Learning to code – and the computational thinking that goes with it – is fun and interesting as well as being intellectually good for you. It’s also potentially lucrative: coding skills are at a premium, wherever you are in the world. While there is still a need for certain people to know Morse Code, there are many other languages to know about: from the languages of data to the logic of a sick (sic) 3D immersive games experience.
I have carried a link to CodeCademy on this site for some time because they offer some excellent resources and courses for people to learn how to code. I have used some of them myself and recommend them highly. If you’re not sure where to start, there is a visual overview of the main programming languages and possible benefits of learning each one to help you make an informed decision. You can find it here: http://wiht.link/learncodeguide.
DISCLAIMER: I am not connected with Codecademy and have received no financial or other incentive to write this post. The infographic is not Codecademy’s and includes links to other free online places where you can learn. It’s just a good idea and a good place to get started. Get on with it!
There’s a lens in every piece of writing and an agenda in most. In George Veletsianos’ Networked Scholars course this week, we are asked to engage with Zeynep Tufekci‘s blog post, which is a piece of emotive writing about another piece of emotive writing in the Grauniad by Emma Keller, about another piece of emotive writing by Lisa Adams, who is blogging about grief and her own battle with cancer.
Each piece takes a stance. Lisa’s stance is perhaps the most authentic as the writing is her own about her own experience. I’m not sure the blog she writes is one I would subscribe to but I understand why she does it: in the same situation, I am likely to be just as loud about it, for at least as long as it is helpful. There must come a time when writing her blog will cease to be relevant to her.
I didn’t find Emma’s article offensive or even critical: I thought she merely asked a question and certainly wasn’t what Zeynep calls “cancer-shaming”. Nor did Emma misrepresent what was happening to Lisa. If there’s fake politically-correct hysteria anywhere here, it’s in Zeynep’s squealing about Emma’s methods. The obtuseness of Zeynep’s complaints is irresponsible for whipping up emotion: for example, her response to Bill Keller’s piece on Lisa – itself tactful, insightful and personal, in my opinion – is disingenuous at best. At worst, it falsifies the content and meaning of what Bill Keller wrote in order to be further outraged.
What is evident in reading these pieces is that social media and blogs are powerful channels through which opinion may be manipulated. Rigour is not required to achieve this as readers, like the baying pitchfork-carrying mobs in a Hammer Horror, respond with such Twitter outrage that the offending item is removed, as in the case of Emma Keller’s article. The Kellers wrote in even tones using moderated language about a woman coping through writing publicly. What Zeynep Tufekci did was to twist that into something very nasty.
This week, one of my sites, sptr.net, has been under a co-ordinated and sustained attack from what appears to be a botnet – a collective of several hundred virus-infected computers running Microsoft Windows. The attack comprises attempts to use the remote procedure call methods built into WordPress to post unauthorised content.
I was notified by one of my independent monitoring services that the site was having trouble some time after the attack began. It appears that once triggered by the attacker, it takes a while for the command to spread to a significant number of infected machines – this is reasonable if you assume the greatest number of infected PCs is in the USA. The attack peaked around the middle of the day in Scotland, coincident with the switching on of computers as the sun moved East to West across the continental US. Although the server remained operational, it was struggling to continue to respond to requests in a reasonable time as the CPU usage soared way above 1000% of nominal maximum. A look at the top processes on the server showed that it was trying to keep things together:
PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command 12345 xxxxxxx 20 0 55992 36080 7128 R 49.0 6.9 0:15.73 [see below] 12346 xxxxxxx 20 0 56036 36124 7188 R 46.0 6.9 0:04.96 [see below] 12347 xxxxxxx 20 0 55940 36076 7128 R 46.0 6.9 0:03.82 [see below] 12349 xxxxxxx 20 0 55912 35908 6984 R 46.0 6.8 0:07.88 [see below] 12340 xxxxxxx 20 0 55976 36116 7180 R 46.0 6.9 0:03.59 [see below] 12342 xxxxxxx 20 0 55940 36064 7128 R 44.0 6.9 0:07.21 [see below] 12341 xxxxxxx 20 0 55948 36140 7196 R 44.0 6.9 0:34.79 [see below] 12343 xxxxxxx 20 0 55972 36248 7276 R 44.0 6.9 2:20.11 [see below]
The command attempted showed that it was an attack on a php script:
/usr/bin/php-cgi -c /var/www/vhosts/sptr.net/etc/php.ini
Looking at the server access logs identified the specific script targeted by the attacker, the machines and methodology involved. The range of IP addresses showed that the infected PCs were world-wide (in the sample below, India, Poland, Egypt, Thailand, Algeria, Brazil and Pakistan).
18.104.22.168 - - [10/Jul/2014:14:03:19 +0000] "POST /xmlrpc.php HTTP/1.1" 200 159 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 22.214.171.124 - - [10/Jul/2014:14:03:34 +0000] "POST /xmlrpc.php HTTP/1.1" 200 159 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 126.96.36.199 - - [10/Jul/2014:14:03:43 +0000] "POST /xmlrpc.php HTTP/1.1" 200 159 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 188.8.131.52 - - [10/Jul/2014:14:03:54 +0000] "POST /xmlrpc.php HTTP/1.1" 200 159 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 184.108.40.206 - - [10/Jul/2014:14:04:04 +0000] "POST /xmlrpc.php HTTP/1.1" 200 159 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 220.127.116.11 - - [10/Jul/2014:14:04:06 +0000] "POST /xmlrpc.php HTTP/1.1" 200 159 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)" 18.104.22.168 - - [10/Jul/2014:14:04:14 +0000] "POST /xmlrpc.php HTTP/1.1" 200 159 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)"
Restarting the VPS container made no difference. CPU usage remained very high. Installing a plugin to disable XML-RPC in WordPress seemed to make things better, probably because of the response time improvement but as the day progressed, the attack seemed to abate and the server was coping better with CPU usage falling below 100% nominal maximum. The log sample above is from today, when the attacks have fallen to a few per minute instead of the hundreds per second on Tuesday. It looks like the botnet is learning that there are robust passwords on the system that will take too long to guess and is giving up.
Brute force solution
I’m not happy with this constant knocking at my door, however, so have decided that I don’t need a door there at all. Removing the target script doesn’t directly affect the rate of attack, it changes the 200 response to a 404 (page not found), which is quickly delivered.
22.214.171.124 - - [10/Jul/2014:14:09:13 +0000] "POST /xmlrpc.php HTTP/1.1" 404 430 "-" "Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)"
Continuing the summer of code into the early autumn, I have been developing, enhancing and debugging the new server. New and migrated sites are stable and responding well within the resource limits I’ve chosen of 10GB disk, 50GB traffic (although we’re close to whacking this one) and 256/512 MB RAM/Swap space. Uptime has been 100% for over 60 days now.
Within the suite of services running on the server are database, web server, CGI, mail, stats and monitoring. What is not, is the DNS service, which I have learned to keep in a different place, with the registrar. Setting up reverse DNS for the mail service to work correctly is important: I discovered that one client had been having difficulties receiving mail from just one of his friends. This was because the MX DNS entry for his domain pointed to an IP address which some service providers will reject as it doesn’t comply with the RFC. Changing it to the host domain of the server’s IP, however, stopped all mail getting through to the client. This was finally resolved by pointing the MX record for the domain to the domain itself:
example.com. A 192.0.2.1 @ MX 10 example.com.
If you want to know how the Internet works, by the way, a really good place to start is the Internet Engineering Task Force (IETF). They have a good introduction here. Many internet standards are defined in RFC documents.
Other services on the server operate as database-driven php suites such as the WordPress CMS, Moodle, LimeSurvey or phpBB. All of these are subject to modifications, code hacks and tweaks to make them work to the needs of the site owner. Whilst the Parallels Plesk Panel allows install-at-a-click for many application suites, I prefer to manage the installation and customisation of these myself. Until now, I had used the download-unzip-upload over FTP method but I’m going to try using the more elegant command-line facility offered by Git. I’m getting started by using their excellent online documentation. This should allow me a much faster update route and potentially a way to be a better contributor to open source than the consumer I have been.
The summer has had me getting to grips with the nitty-gritty of internet web hosting, caused by a consolidation and move of all of the websites and services that I host to a new server. I had been using HostPapa in a shared environment for several years but the traffic and resource usage of these sites had been on the increase for about 18 months, to the point that HostPapa invited me to pack up and leave.
After a detailed survey of requirements and possible alternatives, I elected to move to the affordable but much more powerful next-step-up of a virtual private server (VPS) solution from HostingUK. I’ve known these guys since they set up business in the late 90’s and felt comfortable that I would get good support from the people behind the business. I haven’t been disappointed.
The new server runs CentOS 6.4, a version of the Red Hat Linux operating system and has the usual LAMP features of Apache Web server, mySQL and PHP, with the Parallels Plex 11 management panel.
My development has been firstly in the area of learning how to set it all up using the Plex panel: it’s a very powerful tool but it’s not quite plug-and-play. The DNS for each of the domains on the site is best managed at the registration server using their nameservers: they have redundancy built in and although the VPS can be its own NS, if it goes down for any reason, this can lead to problems with mail transport and SEO indexing. Within the DNS records for each domain, minimum configuration requires appropriate A, MX and CNAME entries as well as TXT or SPF records to stop your mail from being forever consigned to the spam folder.
Further learning has included getting down and dirty with the *nix command line, from basic file operations to examining logs, setting up CRON and managing and installing further packages. I’ve installed Munin to help identify what normal operation looks like. One of the things that my new insight has given me is an appreciation of just how much sustained attack is endured by even the smallest of websites by the likes of Turkish, Chinese, North Korean and other interests. The importance of having decent passwords is underlined when you see 20,000 (yes, twenty thousand) attempts to guess the root password in a single day.
The summer of code has reminded me of what I’m best at, and what I enjoy doing.