Upgrading solr server from 6.6.0 to 7.3.1

I use Apache’s Solr to provide a global search facility on Moodle. Now that my courses have ended for the summer, it’s time to bite the bullet and upgrade the Solr server software from version 6.6.0 which I installed last year, to the current 7.3.1. This turned out to be more straightforward than I feared, and did not require me to touch the PHP solr module that I had to compile from source when I installed it the first time. Here are the steps:

# cd /opt
# wget http://apache.mirrors.nublue.co.uk/lucene/solr/7.3.1/solr-7.3.1.tgz
# tar zxvf solr-7.3.1.tgz
# cp solr-7.3.1/bin/install_solr_service.sh .
# rm -rf solr-7.3.1
# ./install_solr_service.sh solr-7.3.1.tgz -f

Notice the -f flag which tells the script to upgrade an existing installation. The script stops the currently running instance, extracts the new code and starts the instance. A quick check of the admin interface on port 8983 shows the new code running OK, the cores intact, and the client service on Moodle nominal.

EDIT: At present (June 2018) Solr 7 is not supported on Moodle 35. The latest version of the Solr server that works with Moodle 35 is 6.6.4: the instructions above will install Solr 7.

If you want global search to work with Moodle 35, replace “7.3.1” with “6.6.4” and fetch the code using wget from the Apache archive at http://archive.apache.org/dist/lucene/solr/6.6.4/solr-6.6.4.tgz.

TV On Demand

So, we thought we’d catch up on Channel 4’s Humans with some popcorn. On with the Sky box and the telly and press the TV guide. Catch up. Here we go. Microwave is on. Sounds of corn popping.

Sky says we have to contact them because that channel isn’t free to view on demand on the downstairs telly. Weird. We can watch Channel 4 for free on the internet-connected upstairs telly OK, but we wanted to sit with our popcorn, now ready to eat, downstairs.

OK, no problem. XBox. There’s an app.

XBox says it won’t connect to the internet because it wants to do an update first. Sake.

Update eventually installed and restarted and we can sign in and find the app. It needs installed. No problem. Popcorn almost finished now. App installs, but Channel 4 wants us to sign in. Why?

We can’t sign in, we don’t have an account. Why would we have an account for a free-to-air TV channel? It seems we have to visit their website first to create one. Uckfay offhay. Have you people never heard of GDPR?

Popcorn finished. The moment has gone. Casualty it is.

Bring on the AI revolution! Technology is no threat to us.

Open Badges Closed

Well, that was another massive waste of time. For all the hype and hope of having truly transportable awards for training and achievement, backed by Mozilla, no less, the current state of play is pretty disappointing.

I’m running an online resource for some of my students using Moodle, and have been experimenting with badges for minor achievements to help them remain motivated in an unpressurised way, in what is otherwise a highly pressurised programme. Feedback from them has been that the badges are just a bit of fun, really, and not a significant feature of the course. However, talking to one of the students recently revealed that if the badges were to be publicly show-offable, such as on a LinkedIn profile, then they would add a new level of significance that might help them work just that little bit harder to earn these little digital stickers.

Enabling the backpack link within Moodle and trying it for myself (yes, I have awarded myself one of my own badges, for testing purposes obv), I discover that the whole open badges thing is now dead, having been abandoned by Mozilla. Simple issues such as having multiple email accounts within the backpack remain of no interest as the whole idea withers and dies.

Shame. This whole internet thing seemed like a good idea when it was new.

Open Source 1, Microsoft 0 (own goal)

So, I’m sitting down to work on some stuff for my job that is in a Microsoft file format, because that’s what the corporate world uses, when this message appears. I managed to sign in using my corporate id and their subscription.

What ticks me off is that I have already bought and paid for this software. It makes me even more determined to move to open source.

tryGit: it’s the teach, not the tech.

I picked up a link to a free online course from a recent Linux Voice podcast. I tried the course and wasn’t really impressed with it. The interface is nice, with an embedded virtual terminal to let you practice typing in the commands but the pedagogy is pretty weak. There’s nothing in the presentation that indicates that the designers understand how to construct understanding: all this lovely bit of code is doing is rehearsing a list of commands (and there’s even a click-once shortcut if you can’t be bothered actually typing). It’s more of a checklist than a course. It would be easy to turn it into a really effective bit of online learning with the addition of some better structure and graphics, and maybe a little assessment for learning. A shame, really, as it is clearly a loss leader to sell the Code School itself: I am in the market for some good quality online learning in their area, but I’m not likely to look any further at their catalogue.

ownCloud installation on Centos 7

For some time, I’ve wanted to have a calendaring tool independent of Google Calendar, which has become a central tool to my productivity and a source of concern as to how much data profiling results from it.

This afternoon, I installed the open source ownCloud file storage, calendar and contacts suite on my Centos VPS. It was a straightforward exercise:

  • Create a subdomain on the server and switch it to use PHP 5.6. Add /dev/urandom to open_basedir in php settings.
  • Make a data folder behind the web root, chowned to the web user.
  • Create a MySQL database for the ownCloud service.
  • In the web root folder, get the software:

curl -O https://download.owncloud.org/community/owncloud-10.0.3.tar.bz2

  • Check the MD5 hash, chown and extract. Copy the extracted files into the root folder (be careful to include dotfiles, e.g. cp owncloud/* . and cp owncloud/.* .)
  • Visit the domain to configure the installation.

What this server now provides is an independent calendar service, contacts, and secure file storage, at no additional cost and under my own secure control.

Global Search for Moodle on Centos

My students are using a Moodle VLE to access resources and teaching materials and it became evident that some kind of global search function would help them find things quickly, especially later in the programme when they come to write their assignments.

I’m running Moodle on a CentOS 7.3 virtual private server with Plesk Onyx. The server hosts several other sites running WordPress, bespoke PHP and some other bits and pieces including the usual mail services. Some of the containers require the OS-standard PHP5.4 but a recent upgrade to Moodle 3.3 required me to switch the container to PHP 7.0.

Installing Global Search was a little tricky because of the multiple PHP versions running on the server, but I eventually figured it out to these key steps:

Install the Solr Server

$ cd /opt
$ wget http://apache.mirrors.nublue.co.uk/lucene/solr/6.6.0/solr-6.6.0.tgz
$ tar zxvf solr-6.6.0.tgz
$ cp solr-6.6.0/bin/install_solr_service.sh .
$ rm -rf solr-6.6.0
$ ./install_solr_service.sh solr-6.6.0.tgz
$ chkconfig solr on
$ su - solr -c "/opt/solr/bin/solr create_core -c moodle"

You should be able to visit http://your-domain.tld:8983 to verify the Solr server is running OK.

Secure the Solr Server

By default, Solr is open to the world. You might want to secure it by adding this at the end of /opt/solr/server/etc/webdefault.xml:

       <web-resource-name>Solr Administration</web-resource-name>

   <realm-name>Solr Administration</realm-name>

Create a file in the same directory called realm.properties containing your chosen authentication details (matching the role above) in a single line:

admin: password, solr-admin

Finally, add this just before the last line in jetty.xml in the same directory:

<Call name="addBean">
  <New class="org.eclipse.jetty.security.HashLoginService">
    <Set name="name">Solr Administration</Set>
    <Set name="config"><SystemProperty name="jetty.home" default="."/>/etc/realm.properties</Set>
    <Set name="refreshInterval">0</Set>

Install the PHP Solr Extension

$ rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
$ rpm -Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm
$ yum install libxml2-devel pcre-devel libcurl-devel php70w-devel php70w-pear

You’ll need to build the extension using the right versions of phpize and php-config for your version of PHP, in my case, 7.0:

$ cd /opt
$ curl -O https://pecl.php.net/get/solr-2.4.0.tgz
$ tar zxvf solr-2.4.0.tgz
$ cd solr-2.4.0/
$ ../plesk/php/7.0/bin/phpize
$ ./configure --with-php-config=/opt/plesk/php/7.0/bin/php-config
$ make
$ make install
$ cp /opt/solr-2.4.0/modules/solr.so /opt/plesk/php/7.0/lib64/php/modules/
$ sudo service httpd restart

Visit the Site administration / ▶︎ Plugins / ▶︎ Search / ▶︎ Manage global search page in your Moodle installation to configure, index and enable the Solr Search Engine.

I am impressed with how quickly this has been used and appreciated by the students.

get_iplayer broken, version 3.00 available

For those of us who make use of the amazing get_iplayer program to obtain clips and other resources for classroom and other conveniences, it comes as a bit of a blow to find that in the past week or two, it has stopped working. Fortunately, there is a new version of the program available that with a little effort, gets the facility working again.

From the release notes:

The BBC removed all the XML-based data sources used by get_iplayer on 2017-04-26, breaking a lot of get_iplayer functionality. That functionality has been restored, but there are changes to be aware of – get_iplayer has not survived unscathed.

Phil Lewis and the team have (once again) done a fantastic job of quickly responding to changes in the way the BBC delivers its content. Many, many thanks to all the devs and hacks involved in this release.

Finally, my advice to users is to read the release notes carefully. You may also hit issues installing the new dependencies including Mojolicious and Perl as well as the cpan perl repository. Persevere, there is lots of useful advice out there. Finally, finally, the cache updates are much slower than before, although they are now only updated weekly.

Photographer portfolios – Koken

I set up a test site for a photography journal over at http://dev.cullaloe.net/koken/. I’ve been trying a number of alternatives and hosting options: koken is php software that runs on a Linux server over a mySql database and Apache. I happen to have one of those at dev.cullaloe.net.

So far, it looks like it has really nice features, including a tight integration with Adobe Lightroom that allows you to set up a direct publishing link. Most of the images on the site are reduced-size versions of some of my “good” photos.

I have found some bugs and irritations: the admin back-end fails completely from time to time, requiring clearing of api file cache over FTP. Themes are limited but they are quite pretty, I think, with development quite straightforward.

The original developer of this programme sold out to a new owner last year, I believe, but there seems to be some investment in bug fixing and development.

So far I don’t think it’s stable enough for a main online portfolio: you should probably just buy yourself a 500px Awesome membership for that and use the portfolio feature of that site.

Browse safer with a VPN

Image attributed to Duncan Hull (Flickr) and used under the Creative Commons Licence 2.0

Image attributed to Duncan Hull (Flickr) and used under the Creative Commons Licence 2.0

According to a popular Internet Meme, Maslow’s Hierarchy of Needs has a fundamental extension in the modern age – we all seem to need WiFi in order to function.

This need has created a fabulous opportunity for thieves. The term “WiFi” is a synonym for a Wireless (radio) Local Area Network or WLAN. Some WLANs may be password protected but many are not – these “open” networks are found in public spaces, provided by towns or shopping centres to attract trade. In open networks all devices in range can access the network’s resources – i.e. read what you’re typing.

Given that, why would you broadcast your login details, bank data, personal information and private conversations in an open space for all and anyone to listen to, take note of, and make use of at their convenience?

The best defence is to make sure that all internet activity on your mobile devices is sent over an encrypted, secure, connection. The WiFi providers don’t guarantee this, so you have to bring your own in the form of a Virtual Private Network or VPN.

Many corporate employers provide a VPN for staff to ensure that they are not exposed to commercial risk or litigation from data leaks. If that’s not available to you, you can use a service to keep yourself safe: my own personal choice is GoldenFrog’s VyperVPN. It costs a few dollars a year but has fast support and secure servers worldwide. You would normally use a nearby server for speed but if you’re trying to watch the BBC iPlayer in a bar in Prague, you can just connect via the London server to virtually relocate yourself back home to enjoy the latest episode of East Enders. I recommend you don’t use the apps they provide – they don’t work very well and will more often than not just stop you from using the internet. Manual configuration is the way to go: follow the Vyper guide here.

So, a VPN helps keep you safe by encrypting all of your data sent over WiFi; it also works for 3G/4G; and it can virtually relocate you to somewhere you aren’t. What it won’t do is protect you from stupidity – you still have to take care that you aren’t using easy passwords, you don’t use the same password for everything, and you know who’s looking at your screen, for example.

Finally, the image of Maslow’s Hierarchy of Needs above doesn’t really need that big box for WiFi. To be understood in the modern age, we should really put WiFi usage in the Belonging or Self-Esteem levels, but be aware that every user should take action to ensure that Safety is taken care of.