tryGit: it’s the teach, not the tech.

I picked up a link to a free online course from a recent Linux Voice podcast. I tried the course and wasn’t really impressed with it. The interface is nice, with an embedded virtual terminal to let you practice typing in the commands but the pedagogy is pretty weak. There’s nothing in the presentation that indicates that the designers understand how to construct understanding: all this lovely bit of code is doing is rehearsing a list of commands (and there’s even a click-once shortcut if you can’t be bothered actually typing). It’s more of a checklist than a course. It would be easy to turn it into a really effective bit of online learning with the addition of some better structure and graphics, and maybe a little assessment for learning. A shame, really, as it is clearly a loss leader to sell the Code School itself: I am in the market for some good quality online learning in their area, but I’m not likely to look any further at their catalogue.

Migrating from AudioBoom

I’ve been podcasting using Audioboo (now Audioboom) since 2011 and supported the service by paying for a podcaster subscription for much of that time. Audioboom introduced a new pricing structure that I can’t sustain so I need to migrate to a new solution.
Using my LAMP server, which has bandwidth capacity for my podcasts, I set up a new subdomain on http://podcast.cullaloe.net and installed podcastgenerator 2.6. I uploaded a selection of episodes over FTP and used the import function in the admin interface (I had to change the timestamp using touch -t yymmddhhmm.ss filename.mp3 for each episode to get them into the right time order) finally editing in the description for each show.
For the most part, my episodes had embedded images in the mp3 file, but where they did not, I thought about having to edit the ID3 tags. I read about a method for avoiding this which includes posting an image (say, 300×300) for each track within the long description field in the admin interface of the new software, but this doesn’t seem to work. There’s a command line way to pull an image into an .mp3 file which sometimes works as an alternative:
ffmpeg -i input.mp3 -i cover.png -map 0:0 -map 1:0 -c copy -id3v2_version 3 -metadata:s:v title="Album cover" -metadata:s:v comment="Cover (Front)" out.mp3
If that doesn’t work, you can just drop an appropriate jpg/png in the images folder of the same name as the episode audio file.
The iTunes podcast feed was easily updated by visiting https://podcastsconnect.apple.com/  and pointing to the new feed. Note that podcast artwork must now be between 1400 x 1400 and 3000 x 3000 pixels, JPG or PNG, in RGB color space, and hosted on a server that allows HTTP head requests. The iTunes podcast updated overnight without any difficulty.
Finally, I emailed audioboom to request they apply a 301 or iTunes <itunes:new-feed-url/> redirect to the old feed (pointing to the new one).

Global Search for Moodle on Centos

My students are using a Moodle VLE to access resources and teaching materials and it became evident that some kind of global search function would help them find things quickly, especially later in the programme when they come to write their assignments.

I’m running Moodle on a CentOS 7.3 virtual private server with Plesk Onyx. The server hosts several other sites running WordPress, bespoke PHP and some other bits and pieces including the usual mail services. Some of the containers require the OS-standard PHP5.4 but a recent upgrade to Moodle 3.3 required me to switch the container to PHP 7.0.

Installing Global Search was a little tricky because of the multiple PHP versions running on the server, but I eventually figured it out to these key steps:

Install the Solr Server

$ cd /opt
$ wget http://apache.mirrors.nublue.co.uk/lucene/solr/6.6.0/solr-6.6.0.tgz
$ tar zxvf solr-6.6.0.tgz
$ cp solr-6.6.0/bin/install_solr_service.sh .
$ rm -rf solr-6.6.0
$ ./install_solr_service.sh solr-6.6.0.tgz
$ chkconfig solr on
$ su - solr -c "/opt/solr/bin/solr create_core -c moodle"

You should be able to visit http://your-domain.tld:8983 to verify the Solr server is running OK.

Secure the Solr Server

By default, Solr is open to the world. You might want to secure it by adding this at the end of /opt/solr/server/etc/webdefault.xml:

  <security-constraint>
   <web-resource-collection>
       <web-resource-name>Solr Administration</web-resource-name>
       <url-pattern>/*</url-pattern>
   </web-resource-collection>
   <auth-constraint>
       <role-name>solr-admin</role-name>
   </auth-constraint>
  </security-constraint>

  <login-config>
   <auth-method>BASIC</auth-method>
   <realm-name>Solr Administration</realm-name>
  </login-config>

Create a file in the same directory called realm.properties containing your chosen authentication details (matching the role above) in a single line:

admin: password, solr-admin

Finally, add this just before the last line in jetty.xml in the same directory:

<Call name="addBean">
 <Arg>
  <New class="org.eclipse.jetty.security.HashLoginService">
    <Set name="name">Solr Administration</Set>
    <Set name="config"><SystemProperty name="jetty.home" default="."/>/etc/realm.properties</Set>
    <Set name="refreshInterval">0</Set>
  </New>
 </Arg>
</Call>

Install the PHP Solr Extension

$ rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
$ rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
$ yum install libxml2-devel pcre-devel libcurl-devel php70w-devel php70w-pear

You’ll need to build the extension using the right versions of phpize and php-config for your version of PHP, in my case, 7.0:

$ cd /opt
$ curl -O https://pecl.php.net/get/solr-2.4.0.tgz
$ tar zxvf solr-2.4.0.tgz
$ cd solr-2.4.0/
$ ../plesk/php/7.0/bin/phpize
$ ./configure --with-php-config=/opt/plesk/php/7.0/bin/php-config
$ make
$ make install
$ cp /opt/solr-2.4.0/modules/solr.so /opt/plesk/php/7.0/lib64/php/modules/
$ sudo service httpd restart

Visit the Site administration / ▶︎ Plugins / ▶︎ Search / ▶︎ Manage global search page in your Moodle installation to configure, index and enable the Solr Search Engine.

I am impressed with how quickly this has been used and appreciated by the students.

Photographer portfolios – Koken

I set up a test site for a photography journal over at http://dev.cullaloe.net/koken/. I’ve been trying a number of alternatives and hosting options: koken is php software that runs on a Linux server over a mySql database and Apache. I happen to have one of those at dev.cullaloe.net.

So far, it looks like it has really nice features, including a tight integration with Adobe Lightroom that allows you to set up a direct publishing link. Most of the images on the site are reduced-size versions of some of my “good” photos.

I have found some bugs and irritations: the admin back-end fails completely from time to time, requiring clearing of api file cache over FTP. Themes are limited but they are quite pretty, I think, with development quite straightforward.

The original developer of this programme sold out to a new owner last year, I believe, but there seems to be some investment in bug fixing and development.

So far I don’t think it’s stable enough for a main online portfolio: you should probably just buy yourself a 500px Awesome membership for that and use the portfolio feature of that site.

PHP Mail and stripping of lines in Microsoft Outlook

A client recently contacted me about problems with the formatting of messages he was getting from a php contact form on his site. He asked if I could insert a couple of CRLFs to make it easier to read and to stop it breaking the email links in the message.

The client’s site is one of those creaking anachronistic beasts, from the days of hand-hacked HTML, which is full of things that work just well enough to enable him to concentrate on his business. I’ve been trying to get him to move to a CMS like WordPress for several years now, but he’s not quite able to let go.

The contact form had not been a problem, as far as I knew, but all this while he has been putting up with messages from the site that look a bit like this:

Name: FredEmail: fred@bloggs.comTel: 09999899988Hi I was
wondering blah blah blah blah?RegardsFred

On my machines, they look like this:

Name: Fred
Email: fred@bloggs.com
Tel: 09999899988
Hi I was wondering blah blah blah blah?
Regards
Fred

It seems that there is a “feature” that has existed in Microsoft Outlook since 2002, at least. What it does, often without letting the user know, is strip out any formatting of lines in the original message and replaces it with what it thinks you’d prefer. In text-only messages, this results in what you see in the first example above.

There’s a lot written about this, much of it along the lines of altering the user’s practice to include workarounds that are only necessary because Microsoft can’t write good code. See here, for example, or here for one of the empirical solutions that suggests changing code to accommodate Outlook’s perverse behaviour. Many others remain baffled. However, thanks to a bit of forensic inquiry by Matthew Truesdell, there are some rules that can be interpreted in such a way that allows the php script to work for all users. Matthew posted the rules he found in Outlook 2007, over on Stack Overflow: I’ve adapted from those here, slightly, using the term “mode” to mean the behaviour of Outlook that strips out line breaks from plain text messages. Lines are assessed one at a time:

  • Every message starts with the mode OFF.
  • Lines 40 characters or longer switch the mode ON.
  • Lines that end with a full stop (.), question mark (?), exclamation (!) or colon (:) switch the mode OFF.
  • Lines that turn the mode off will start with a line break, but will turn it back on if they are longer than 40 characters.
  • Lines that start or end with a tab turn the mode off.
  • Lines that start with 2 or more spaces turn the mode off.
  • Lines that end with 3 or more spaces turn the mode off.

So it seems that one way to trick Outlook is to add 3 spaces at the end of each line, which in the code is just before the CRLF. I tried this, but be careful if you rely on it: different versions of Outlook do different things. Outlook 2013 is still stripping out the line breaks on the client machine, so we have this:

Name: Fred   Email: fred@bloggs.com   Tel: 09999899988
Hi I was wondering blah blah blah blah?   Regards   Fred

Which is still not satisfactory but at least allows him to click on the email address for a quicker response.

On my own machine (OSX Yosemite), Outlook 10 seems to be working as you’d expect, without interfering with the line breaks. Gmail works fine also. I think that’s as far as I’m going to take it.

CentOS 6.5 on MSWind

Rather than make a pig’s ear out of my live VPS by testing out new Ruby code I’m playing with, I thought it would be prudent to have a machine that I can break without upsetting users. I have an Atom-based Advent netbook which only ever gets played with occasionally and this afternoon, seems quite willing to volunteer for a rebuild as a CentOS server. The world loves a volunteer. Continue reading “CentOS 6.5 on MSWind”

Metamorphosis of Narcissus

MetaNarcissus was, according to legend, a hunter. Walking in the woods, he was seen by the nymph Echo, who falls in love with him. The nymph had been cursed by Hera, the wife of Zeus, such that she could only repeat the last words heard and not say anything of her own. Narcissus rejects Echo’s love. After praying to Aphrodite, she disappears, remaining only as a voice heard by all.

The goddess of revenge, Nemesis, punishes Narcissus by leading him to fall in love with his own image reflected in a spring. Different outcomes, none of them good, await Narcissus, depending on the version of the story you read. Continue reading “Metamorphosis of Narcissus”

Pro Git and more MX DNS

Continuing the summer of code into the early autumn, I have been developing, enhancing and debugging the new server. New and migrated sites are stable and responding well within the resource limits I’ve chosen of 10GB disk, 50GB traffic (although we’re close to whacking this one) and 256/512 MB RAM/Swap space. Uptime has been 100% for over 60 days now.

Within the suite of services running on the server are database, web server, CGI, mail, stats and monitoring. What is not, is the DNS service, which I have learned to keep in a different place, with the registrar. Setting up reverse DNS for the mail service to work correctly is important: I discovered that one client had been having difficulties receiving mail from just one of his friends. This was because the MX DNS entry for his domain pointed to an IP address which some service providers will reject as it doesn’t comply with the RFC. Changing it to the host domain of the server’s IP, however, stopped all mail getting through to the client. This was finally resolved by pointing the MX record for the domain to the domain itself:

example.com. A     192.0.2.1
@            MX 10 example.com.

If you want to know how the Internet works, by the way, a really good place to start is the Internet Engineering Task Force (IETF). They have a good introduction here. Many internet standards are defined in RFC documents.

Other services on the server operate as database-driven php suites such as the WordPress CMS, Moodle, LimeSurvey or phpBB. All of these are subject to modifications, code hacks and tweaks to make them work to the needs of the site owner. Whilst the Parallels Plesk Panel allows install-at-a-click for many application suites, I prefer to manage the installation and customisation of these myself. Until now, I had used the download-unzip-upload over FTP method but I’m going to try using the more elegant command-line facility offered by Git. I’m getting started by using their excellent online documentation. This should allow me a much faster update route and potentially a way to be a better contributor to open source than the consumer I have been.