Thoughts on the “Spring Hunting” vote..

Lets preface this with the line: I hunt and shoot and would vote “NO” if I could vote…

I will still hunt and shoot even if the “NO” vote wins. Hunting and Shooting as I do takes a lot of skill and is very demanding. My weapon of choice is different from the average hunter though (which is what makes it a lot harder)… Here’s the results of my shooting:

Maltese hunters target passage Spoonbills

Mark Sultana also shot a few and the Times of Malta posted the results of his hunting… (which if you click the photo you’ll see more and you will see the skill level is so high that when in flight getting the results is significantly more difficult.)

Spoonbills shot, BirdLife to protest to EU after Spring hunting decision

Oddly enough it seems there are many like me that enjoy hunting and shooting, and we can quite legitimately and legally hunt and shoot protected birds as well as non protected birds…

Pity the “YES” vote chooses to show people crossing fields with not a bird in sight, after all isn’t that what they are in favour of?

Pity the “NO” vote feels the need to post all the blood and gore of birds that have been blasted out of the sky being carried by tree huggers and the likes.

Pity the church (which seems to play politics here in Malta on items like Same-Sex marriage whenever it feels like it, yet) don’t seem to care that a “YES” vote means legitimising the killing of God’s Creations FOR FUN….

When it comes to voting on 11 April, don’t vote how people tell you to vote, don’t allow people to tell you how to vote, vote with your conscience… i.e. what you really feel is right, if that means vote “YES” then do it without worry, if that means “NO” then vote “NO”, the point of a referendum is to allow *YOU* to choose for yourself!

(And for those people whom their family and/or friends are being pressured into voting “YES” or “NO” just remember when it comes to voting, it is your choice, tell them what they want to hear, then when you get into the booth, tick the box *YOU* want to vote for. Remember, it’s YOUR vote not THEIRS!)

Converting h.265 (HEVC) to h.264 (AVC)

Quick techie entry for anyone using the newer h265 codecs but unable to use them in players. (eg: Torrenting H265 encoded files then trying to play via PLEX and Roku)

Roku and other media players don’t support h.265 and as such any attempt to play h265 encoded files will result in an ‘Unable to play file’ error, so you might be wanting to convert the files to another format such as h.264.  To do this you need ffmpeg, however ffmpeg can be a little difficult to work, especially as it has so many options, so I wrote a little perl script to ‘mass convert’ all files in the current directory if they are h265 encoded to h264 encoding.  It is published here for those on UNIX systems (or those who know how to install Perl on Windows) to make life a little easier:

#!/usr/bin/perl

use strict;
use warnings;

open DIR, "ls -1 |";
while (<DIR>)
{
        chomp;
        next if ( -d "$_"); # skip directories
        next unless ( -r "$_"); # if it's not readable skip it!
        my $file = $_;
        open PROBE, "ffprobe -show_streams -of csv '$file' 2>/dev/null|" or die ("Unable to launch ffmpeg for $file! ($!)");
        my ($v, $a, $s, @c) = (0,0,0);
        while (<PROBE>)
        {
                my @streaminfo = split(/,/, $_);
                push(@c, $streaminfo[2]) if ($streaminfo[5] eq "video");
                $a++ if ($streaminfo[5] eq "audio");
                $s++ if ($streaminfo[5] eq "subtitle");
        }
        close PROBE;
        $v = scalar @c;
        if (scalar @c eq 1 and $c[0] eq "ansi")
        {
                warn("Text file detected, skipping...\n");
                next;
        }
        warn("$file: Video Streams: $v, Audio Streams: $a, Subtitle Streams: $s, Video Codec(s): " . join (", ", @c) . "\n");
        if (scalar @c > 1)
        {
                warn("$file has more than one video stream, bailing!\n");
                next;
        }
        if ($c[0] eq "hevc")
        {
                warn("HEVC detected for $file ...converting to AVC...\n");
                system("mkdir -p h265");
                my @params = ("-hide_banner", "-threads 2");
                push(@params, "-map 0") if ($a > 1 or $s > 1 or $v > 1);
                push(@params, "-c:a copy") if ($a);
                push(@params, "-c:s copy") if ($s);
                push(@params, "-c:v libx264 -pix_fmt yuv420p") if ($v);
                if (system("mv '$file' 'h265/$file'"))
                {
                        warn("Error moving $file -> h265/$file\n");
                        next;
                }
                if (system("ffmpeg -xerror -i 'h265/$file' " . join(" ", @params) . " '$file' 2>/dev/null"))
                {
                        warn("FFMPEG ERROR.  Cannot convert $file restoring original...\n");
                        system("mv 'h265/$file' '$file'");
                        next;
                }
        } else {
                warn("$file doesn't appear to need converting... Skipping...\n");
        }
}
close DIR;


Enjoy!

A history of my experience with FreeBSD and a warning to users….

So a rant about how a great project can go bad, and how it’s still doing stuff that should never be done and why I’ve decided to ‘fix it myself’ or move away from it completely.

Back in 2003 Paul Vixie forced me into using FreeBSD on one of my servers, it was not a welcome change for me, I was an avid Linux user until this point…. and it didn’t go well.. I started on 4.x, found there was no threading support, so “upgraded” the system to 5.x… which went badly…very badly… and every upgrade through 5.x was as bad…. Partly because of what I did, partly because of my lack of knowledge and partly because of system limitations.

Out came 6.0 and I started working with it and soon I had a whole slew of machines that were on 6.0 and with 6.1 and 6.2 things only got better. I had build servers, I had package servers, I could boot one of the servers off the network and have it completely re-install the server with the latest OS, Patches and Packages within 23 minutes (bare metal to built, configured and in production in 23 minutes…!)

Then came 7.0 and my ‘burnout’ – personal, profession life clashed, I ‘burnt out’ and my technical issues took a back burner, then before you knew it 7.3 and 7.4 were out and I had sold my company… and I was back working on getting things patched and upgraded… however some major changes had happened and the ‘ports’ tree no longer worked on 6.x hosts… so the entire system was frozen…. no new security patches, no upgrades, however along with the sale of the company came new hope… new hardware… and an opportunity to upgrade by replacing the hardware… New hardware was installed to 7.3 (as this was all that was available on Softlayer) and then my attention was diverted to getting my software upgraded to a new major revision and with it my attention and priorities changed from Sys-Admin work to developer and the older systems remained. Not long later the company that ‘bought me’, ‘sold me’ to another (my current) employer, Proofpoint Inc and new priorities were given… along with more new servers.. the result was 8.x systems being installed and with the advent of FreeBSD upgrading ‘bmake’ more stuff got changed in the ports tree, again making them non-working on pre 7.4 systems… more things got changed/patched on my servers and I ended up with new hardware again, this time running 9.0 and 9.1… at this point in time (2013) I had the following versions of FreeBSD in production:

  • FreeBSD 6.0
  • FreeBSD 6.1
  • FreeBSD 6.2
  • FreeBSD 7.2
  • FreeBSD 7.3
  • FreeBSD 8.1
  • FreeBSD 8.2
  • FreeBSD 8.3
  • FreeBSD 8.4
  • FreeBSD 9.0
  • FreeBSD 9.1

Which for any sys-admin you can guess would be a nightmare.  Further Proofpoint has policy and puppet, policies about how things are managed and puppet to manage everything.  It was suggested that my systems should be managed by puppet… so after Oct 2013 when the databases were finally migrated to the new hardware and then I could work on upgrading everything off old hardware and onto new OS’s and patches I setup a puppet server, a number of build servers and a test suite, all of my own creation and similar to what I had done in 2005… to take back control…  I also ended up with FreeBSD 9.2 on some servers, so I decided i would standardise on:

  • FreeBSD 8.4
  • FreeBSD 9.0
  • FreeBSD 9.1
  • FreeBSD 9.2

…at least until I could spend the time getting everything to a single OS level…  FreeBSD 10.0 came out, and later FreeBSD 9.3, but by that time I had the basic systems working and so adding these to the build and test suite was a matter of adding new build and test hosts… which just took a few hours.

As part of this build change I learned new tools:

  • Jenkins
  • Poudriere
  • Puppet
  • VirtualBox

I learned how to create my own ports, I learned how to patch my own ports privately.  I learned how to submit bugs back to FreeBSD ports maintainers.  I became a FreeBSD port maintainer myself.  I noted that as of 1st September 2014 the old pkg_* tools that had been around since day dot were about to be End-Of-Life’d in favour of a new ‘PKGNG‘ system.  I read the linked blog entry and decided that it was something I would have to look at, but later, because the EOL (as most sys-admins know) just means no new patches and something may start breaking that wouldn’t be supported by the developers.  At the end of July 2014 I spoke with the main protagonist of the change and was informed bluntly and to the point that they had already got a patch built and waiting to be applied, not to EOL the tools but to actually and deliberately break the existing tools thereby forcing people to use the new system.

Needless to say with less than 5 weeks of time before this was due to occur there was no chance of me converting all 57 servers, so I suggested that they shouldn’t I was told, its going to happen regardless… and that I should know that EOL means the product would no longer work, not that it would just not be supported anymore.  I guess all those years I had worked for the likes of Netscape, Oracle etc meant they all got it wrong… even Microsoft got it wrong, I mean Windows XP was ‘EOL’d a while back and well all those Window XP machines around the world just stopped working the same day… NOT!

So I continued with my build system and tried to get a stable patched repository of packages so I could at least continue my plan to get the servers to the standardised OS levels…  During testing of the packages I noted bugs, reported them to the developers, then pushed the maintainers (with mixed levels of success) to implement the fixes before the dead line (more appropriately named rather than EOL)… I failed.. several patches were not put into the ports tree until 7 days after the dead line (and that may have been deliberate on the developers aspect – though will never know.)  So the ports tree was patched on time, it rendered the old tools dead and my entire build, test and development system was broken.

I set about repairing it, for a while just copying pre-DeadLine files for building seemed to work with some local changes, so I continued to build out my systems to cope with this, and finally at the beginning of Dec 2014 I got a stable and complete repository.

Over Christmas 2014 I set myself the task of upgrading all servers to one of the standardised OS’s and at the same time patching all the existing OS’s on one of those versions.  Of 57 servers, 31 became un-usable in some way during the patch update process (freebsd-update)  Some became un-bootable, some couldn’t access the network, some (even going from 9.3-RELEASE to 9.3-P5) broke packages such as ‘sudo’ leaving me unable to gain increased privileges to finish the patch process…. after over 160 hours of work, only stopping Christmas day and New Years day, all systems were patched to 9.2 or 9.3 with all the security patches…as they had to be because of the NTPd remote root exploit…. only having to reinstall 2 of the systems from scratch as they were un-recoverable.

Early January 2015 the build system failed again when trying to patch new security issues and I found it was related to more changes by the same culprit so decided after seeing similar rants by other long standing advocates to ask for some help and got a working set of Mk/* files with the intention of fixing it again.  The files I got wouldn’t work so I merged the tree by hand (27900+ lines) only to find the system not quite working… a week later and I have a working build system for most of the ports.  I set it going and get a working repository and decide to re-run the build because of a failed patch, and it all broke again…

So for the warning to all FreeBSD Users:

IF YOU RUN PRODUCTION SERVERS THAT REQUIRE TESTING AND STABILITY BEFORE MAJOR CHANGES, YOU PROBABLY ARE STILL ON PKG_* TOOLS, DON’T UPGRADE, DON’T PATCH AND LOOK AT OTHER SOLUTIONS! Here’s why:

  • running ‘freebsd-update‘ the extra pass to “delete old” will delete all pkg_* tools (even if you haven’t converted to pkgng)
  • updating the ports tree and updating something will automatically convert the system to use pkgng (whether tested and working or not)
  • if you build your own packages using poudriere 3.1 or above it will also “upgrade” your system without confirmation or warning.

Basically whether tested or not, whether working or not, the FreeBSD developers (not the kernel devs as far as I know) will change your production systems to configurations that will probably render your automated systems completely ineffective, without warning and without notification.

What am I doing about it, well at the moment I have created a ports tree ( svn co http://svn.sorbs.net/repos/ports/head ) on http://svn.sorbs.net/repos/ports that can be put into poudriere (as SVN_HOST=svn.sorbs.net/repos ) and it will in theory build most packages for pkg_* tools – it’s not complete and is being changed on a daily basis currently as new changes go in, and with the latest “HEADSUP” announced on the FreeBSD Ports mailing list detailing another change in syntax that is not backward compatible with existing systems (even pkgng ones) I expect it won’t work for long….  My advice as the culprit seems hell bent on changing systems to the way Linux has been for years and ignoring all input from users of FreeBSD that does not agree with his vision, find an alternative.

After 12 years of promoting FreeBSD I am not any more, I’m not going to stop my employer moving everything to Linux, and I’m *NOT* going to upgrade anything to 10.x (and as 9.4 will probably not have pkg_* tools available, I won’t be going there either.)

Sadly, thinking about the whole issue, with a little work it could have been avoided, ensuring all variables in the ports are backwards compatible and having separate Mk/* repositories (even unmaintained/EOLd) would have made the whole process less painful an allowed the developers to continue their path, whether right or wrong, to completion, and allow us insignificant users to continue without pain.  In fact had someone had the for-sight I think even pre-bmake systems would still be patchable and working, even back to the 6.x tree! .. well at least until the new changes in the plist files… which most can be back-ported despite the claim that progress is impossible with the old pkg_* tools.

HBSC Swieqi the target of tunnellers a second time!

Following a number of complaints about a new generator that appeared and has been running non stop for 3 day Swieqi local council ordered an investigation into its purpose and uncovered a government plot to get HSBC Malta to pay for their electricity bills.

Unidentified Plant that "Appeared"
Unidentified Plant that “Appeared”

Council investigators were called to investigate why a new piece of machinery had been surreptitiously installed on a piece of undeveloped ground near the new Swieqi Police station and left running spewing thick diesel smoke into the local area.  Investigators found the plant was powering tunnelling machinery building the new Malta-Sicily Electricity inter-connector.  This publication though can now reveal that the weekend work has been to secretly re-deploy the tunnelling machines on a ‘side project’ and tunnel to into the vault of the HSBC Bank on the St Andrews Road, Swieqi in a daring attempt a government bank robbery with the striking similarity to the recent amateur attempt exposed just last year in 2013.

It is believed this government scheme has been actioned to pay off the mounting energy bills of the ruling parties.  Whether this is a PL, PN or both scheme this publication is unsure as a recent article on the Malta Independent indicate that the PL (Party Labour) owe around 600,000 Euros whilst the PN (Party National) owe in excess of 1,900,000 Euros for their energy consumption.  One also has to consider that the tunnelling project was started under a PN government and during the recent elections the PL gained power and took over the project.  Of course we also know that

Police Uncover Tunnel Entrance.
Police Uncover Tunnel Entrance.

Enemalta, WSC and ARMS has debts in excess of 30,000,000 Euros so this also could be a raid to pay off it’s debts before the up and coming contract signing with the Chinese corporations buying into the energy supply company of Malta… only time will tell.  As of 14:30 today there has been no sign of police investigations despite country wide concern over the alleged robbery attempt, and with the recent change in Police Commissioners it seems more questions are raised than answered.

Our investigators are currently attempting to secure evidence to the organisers, but with Sicily just a small boat ride away our staff have been told to be careful about any questions they ask in the event that this is mob related and personal safety becomes an issue.

€500.00 Reward offered…

For several weeks now, cars on Triq, is-Swieqi, Triq, is-Sirk and Triq, il-Hemel all in Swieqi, near St Julians have been deliberately vandalised, including broken wing mirrors, slashed tyres and ‘Keying’ paint work.

I am therefore offering a €500.00 reward for information leading to the arrest and conviction of the perpetrators.

If you know who is doing this, why or when they are about to do it again (so they can be caught in the act) you should contact the St Julian’s police station on: +356 2137 1694 or myself via Email: michelle@sorbs.net or myself via SMS/Phone: +356 77 543 114.

To claim the reward, you will need to have given me details of when and whom in advance, or you will need to accompany me and make a statement to the police which results in successful prosecution of the offenders.