nortc for raspberry pi …doing it the Fedora way

Previously, I blogged about adding support for Raspberry Pi’s non-existent hardware clock. I have created a simple script that worked as a …err work-around to remind the Pi of the time that it should be on …not 1970! So, I started thinking about testing it in other Raspberry Pi-s, that meant packaging it. I am a Fedora guy so I used Fedora’s packaging app, RPM. I will not get into details regarding the RPM creation process. This link can provide some good information for those who wants to learn. As I already had a 64-bit box setup as my devel box, I transferred the script and the unit file from the Pi to the devel box and created a tarball. I made a simple spec file that could do a basic installation with a unit file in the right location. When I was creating the unit file, I had to learn about systemd and how systemd support can be added in an rpm package. Since I created the nortc script from scratch, I also had to figure out the dependency requirements (both build time and run time) . Once I had those addressed, this is what my spec file looked like:

——————————————————————————–

Name:           nortc
Version:        0.1
Release:        1%{?dist}
Summary:        time and date fix for devices with no RTC
License:        GPLv2+
Group:          System Environment/Base
URL:            http://zenit.senecac.on.ca/wiki/index.php/%{name}
Source0:        http://scotland.proximity.on.ca/maxam/%{name}/%{name}-%{version}.tar.gz
BuildArch:      noarch
BuildRequires:  systemd-units
Requires:       coreutils e2fsprogs gawk grep util-linux systemd-units
Requires(post): systemd-units
Requires(preun): systemd-units
Requires(postun): systemd-units

%description
Fixes the time and date in devices with no realtime clocks,
e.g., virtual machines, embedded systems, specific ARM devices, etc.

%prep
%setup -q

%build

%install
rm -rf %{buildroot}
mkdir -p %{buildroot}%{_sbindir}/
cp nortc %{buildroot}%{_sbindir}/
mkdir -p %{buildroot}%{_libdir}/systemd/system/
cp nortc*.service %{buildroot}%{_libdir}/systemd/system/

%post
if [ $1 -eq 1 ] ; then 
    # Initial installation 
    /bin/systemctl daemon-reload >/dev/null 2>&1 || :
fi

%preun
if [ $1 -eq 0 ] ; then
    # Package removal, not upgrade
    /bin/systemctl --no-reload disable nortc.service > /dev/null 2>&1 || :
    /bin/systemctl stop nortc.service > /dev/null 2>&1 || :
fi

%postun
/bin/systemctl daemon-reload >/dev/null 2>&1 || :
if [ $1 -ge 1 ] ; then
    # Package upgrade, not uninstall
    /bin/systemctl try-restart nortc.service >/dev/null 2>&1 || :
fi

%files
%defattr(0644,root,root,-)
%doc AUTHORS COPYING ChangeLog KNOWN_BUGS NEWS README TODO THANKS
%attr(0755,root,root) %{_sbindir}/*
%{_libdir}/systemd/system/*

%changelog
* Wed Aug 01 2012 Some User <user@domain.tld> - 0.1-1
- initial build for nortc 0.1

——————————————————————————–

I ran rpmlint against the spec file and got the following warning:

——————————————————————————–

[maxam@box SPECS]$ rpmlint -i nortc.spec
 nortc.spec:56: W: libdir-macro-in-noarch-package (main package) %{_libdir}/systemd/system/*
 The %{_libdir} or %{_lib} macro was found in a noarch package in a section
 that gets included in binary packages. This is most likely an error because
 these macros are expanded on the build host and their values vary between
 architectures, probably resulting in a package that does not work properly on
 all architectures at runtime. Investigate whether the package is really
 architecture independent or if some other dir/macro should be instead.
0 packages and 1 specfiles checked; 0 errors, 1 warnings.

——————————————————————————–

It came up due to systemd using the libraries that are acrhitecture specific in a package that was supposed to be architecture independent. I chose to ignore it as it was a non-issue.

Then came the biggest hurdle (at least for me), documentation! Believe it or not, documentation took twice as much time, focus and dedication than creating the script and researching for the spec file. My first attempt to write docs was beyond pitiful; but eventually I managed to come up with something usable. If I start writing about docs, it will take another 20 blog posts, umm… not happening! So, I had the spec file and a new tarball (now with the docs included) to build an rpm package. I followed rpm processing guidelines to create the rpm and the source rpm. These are some of the commands I ran and their output:

——————————————————————————–

[maxam@box ~]$ rpmdev-wipetree
 Removing all build files...
 [maxam@box ~]$ cp ~/nortc/nortc.spec ~/rpmbuild/SPECS/
 [maxam@box ~]$ cp ~/nortc/nortc-0.1.tar.gz ~/rpmbuild/SOURCES/
 [maxam@box ~]$ rpmbuild -ba ~/rpmbuild/SPECS/nortc.spec
 Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.HA5j9D
 + umask 022
 + cd /home/maxam/rpmbuild/BUILD
 + LANG=C
 + export LANG
 + unset DISPLAY
 + cd /home/maxam/rpmbuild/BUILD
 + rm -rf nortc-0.1
 + /usr/bin/bzip2 -dc /home/maxam/rpmbuild/SOURCES/nortc-0.1.tar.gz
 + /usr/bin/tar -xf -
 + STATUS=0
 + '[' 0 -ne 0 ']'
 + cd nortc-0.1
 + /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
 + exit 0
 Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.FecHUK
 + umask 022
 + cd /home/maxam/rpmbuild/BUILD
 + cd nortc-0.1
 + LANG=C
 + export LANG
 + unset DISPLAY
 + exit 0
 Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.2XlaIR
 + umask 022
 + cd /home/maxam/rpmbuild/BUILD
 + '[' /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64 '!=' / ']'
 + rm -rf /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64
 ++ dirname /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64
 + mkdir -p /home/maxam/rpmbuild/BUILDROOT
 + mkdir /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64
 + cd nortc-0.1
 + LANG=C
 + export LANG
 + unset DISPLAY
 + rm -rf /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64
 + mkdir -p /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64/usr/sbin/
 + cp nortc /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64/usr/sbin/
 + mkdir -p /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64/usr/lib64/systemd/system/
 + cp nortc-late.service nortc.service /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64/usr/lib64/systemd/system/
 + /usr/lib/rpm/find-debuginfo.sh --strict-build-id /home/maxam/rpmbuild/BUILD/nortc-0.1
 + '[' noarch = noarch ']'
 + case "${QA_CHECK_RPATHS:-}" in
 + /usr/lib/rpm/check-buildroot
 + /usr/lib/rpm/redhat/brp-compress
 + /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip
 + /usr/lib/rpm/brp-python-bytecompile /usr/bin/python 1
 + /usr/lib/rpm/redhat/brp-python-hardlink
 + /usr/lib/rpm/redhat/brp-java-repack-jars
 Processing files: nortc-0.1-1.fc17.noarch
 Executing(%doc): /bin/sh -e /var/tmp/rpm-tmp.O6qF7Y
 + umask 022
 + cd /home/maxam/rpmbuild/BUILD
 + cd nortc-0.1
 + DOCDIR=/home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64/usr/share/doc/nortc-0.1
 + export DOCDIR
 + /usr/bin/mkdir -p /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64/usr/share/doc/nortc-0.1
 + cp -pr AUTHORS COPYING ChangeLog KNOWN_BUGS NEWS README TODO THANKS /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64/usr/share/doc/nortc-0.1
 + exit 0
 Requires(interp): /bin/sh /bin/sh /bin/sh
 Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
 Requires(post): /bin/sh systemd-units
 Requires(preun): /bin/sh systemd-units
 Requires(postun): /bin/sh systemd-units
 Requires: /bin/bash
 Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64
 Wrote: /home/maxam/rpmbuild/SRPMS/nortc-0.1-1.fc17.src.rpm
 Wrote: /home/maxam/rpmbuild/RPMS/noarch/nortc-0.1-1.fc17.noarch.rpm
 Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.9uBoym
 + umask 022
 + cd /home/maxam/rpmbuild/BUILD
 + cd nortc-0.1
 + /usr/bin/rm -rf /home/maxam/rpmbuild/BUILDROOT/nortc-0.1-1.fc17.x86_64
 + exit 0
 [maxam@box ~]$ tree ~/rpmbuild/
 /home/maxam/rpmbuild/
 ├── BUILD
 │   └── nortc-0.1
 │   ├── AUTHORS
 │   ├── ChangeLog
 │   ├── COPYING
 │   ├── debugfiles.list
 │   ├── debuglinks.list
 │   ├── debugsources.list
 │   ├── KNOWN_BUGS
 │   ├── NEWS
 │   ├── nortc
 │   ├── nortc-late.service
 │   ├── nortc.service
 │   ├── README
 │   ├── THANKS
 │   └── TODO
 ├── BUILDROOT
 ├── RPMS
 │   ├── noarch
 │   │   └── nortc-0.1-1.fc17.noarch.rpm
 │   └── x86_64
 ├── SOURCES
 │   └── nortc-0.1.tar.gz
 ├── SPECS
 │   └── nortc.spec
 └── SRPMS
 └── nortc-0.1-1.fc17.src.rpm
9 directories, 18 files

——————————————————————————–

Since I used a 64-bit box, the binary rpm file I got was for 64-bit Fedora boxes; but I wanted be sure that it built for a 32-bit box and more specifically for an ARM board. Technically, it was supposed to, being a noarch package; but I wanted to test anyways, that meant testing it with mock. Mock checks the build requirements for packages and it works with any source rpm (srpm) file. As it happened, when I used the rpmbuild utility, it created a source rpm file and all I had to do was provide mock with the srpm file and specify which architecture I wanted it to build for. To see what architecture mock supports, mock config files can be found in Fedora’s default configuration location under the mock directory. Here is a snippet of what I did with mock:

——————————————————————————–

[maxam@box ~]$ mock -r fedora-17-x86_64 ~/rpmbuild/SRPMS/nortc-0.1-1.fc17.src.rpm 
INFO: mock.py version 1.1.26 starting...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
INFO: Start(/home/maxam/rpmbuild/SRPMS/nortc-0.1-1.fc17.src.rpm) Config(fedora-17-x86_64)
Start: lock buildroot
Start: clean chroot
INFO: chroot (/var/lib/mock/fedora-17-x86_64) unlocked and deleted
Finish: clean chroot
Finish: lock buildroot
Start: chroot init
Start: lock buildroot
Mock Version: 1.1.26
INFO: Mock Version: 1.1.26
INFO: calling preinit hooks
INFO: enabled root cache
INFO: root cache aged out! cache will be rebuilt
INFO: enabled yum cache
Start: cleaning yum metadata
Finish: cleaning yum metadata
INFO: enabled ccache
Start: device setup
Finish: device setup
Start: yum update
Finish: yum update
Start: creating cache
Finish: creating cache
Finish: lock buildroot
Finish: chroot init
INFO: Installed packages:
Start: build phase for nortc-0.1-1.fc17.src.rpm
Start: device setup
Finish: device setup
Start: build setup for nortc-0.1-1.fc17.src.rpm
Finish: build setup for nortc-0.1-1.fc17.src.rpm
Start: rpmbuild -bb nortc-0.1-1.fc17.src.rpm
Finish: rpmbuild -bb nortc-0.1-1.fc17.src.rpm
Finish: build phase for nortc-0.1-1.fc17.src.rpm
INFO: Done(/home/maxam/rpmbuild/SRPMS/nortc-0.1-1.fc17.src.rpm) Config(fedora-17-x86_64) 1 minutes 15 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-17-x86_64/result
Finish: run
[maxam@box ~]$ mock -r fedora-17-i386 ~/rpmbuild/SRPMS/nortc-0.1-1.fc17.src.rpm 
INFO: mock.py version 1.1.26 starting...
Start: init plugins
INFO: selinux enabled
Finish: init plugins
Start: run
INFO: Start(/home/maxam/rpmbuild/SRPMS/nortc-0.1-1.fc17.src.rpm) Config(fedora-17-i386)
Start: lock buildroot
Start: clean chroot
INFO: chroot (/var/lib/mock/fedora-17-i386) unlocked and deleted
Finish: clean chroot
Finish: lock buildroot
Start: chroot init
Start: lock buildroot
Mock Version: 1.1.26
INFO: Mock Version: 1.1.26
INFO: calling preinit hooks
INFO: enabled root cache
INFO: root cache aged out! cache will be rebuilt
INFO: enabled yum cache
Start: cleaning yum metadata
Finish: cleaning yum metadata
INFO: enabled ccache
Start: device setup
Finish: device setup
Start: yum update
Finish: yum update
Start: creating cache
Finish: creating cache
Finish: lock buildroot
Finish: chroot init
INFO: Installed packages:
Start: build phase for nortc-0.1-1.fc17.src.rpm
Start: device setup
Finish: device setup
Start: build setup for nortc-0.1-1.fc17.src.rpm
Finish: build setup for nortc-0.1-1.fc17.src.rpm
Start: rpmbuild -bb nortc-0.1-1.fc17.src.rpm
Finish: rpmbuild -bb nortc-0.1-1.fc17.src.rpm
Finish: build phase for nortc-0.1-1.fc17.src.rpm
INFO: Done(/home/maxam/rpmbuild/SRPMS/nortc-0.1-1.fc17.src.rpm) Config(fedora-17-i386) 1 minutes 22 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-17-i386/result
Finish: run
[maxam@box ~]$ mock -r fedora-17-arm ~/rpmbuild/SRPMS/nortc-0.1-1.fc17.src.rpm 
ERROR: Cannot build target armv5tel on arch x86_64

——————————————————————————–

So, it couldn’t build for ARM in my x86_64 box; the reason for that was simple, it is completely different from the Intel 32/64-bit architectures! Fortunately, Fedora has a way to deal with it, it is called Koji. Koji is Fedora’s build system, clusters of boxes in remote locations that are capable of  doing the mock builds. Fedora has different Koji instances for different architectures. More information on Koji can be found here. So, for my purpose, I needed to send my srpm file to arm-koji. Since I was just testing it, I sent mine as a scratch build:

——————————————————————————–

 [maxam@box ~]$ arm-koji build f17 --scratch ~/rpmbuild/SRPMS/nortc-0.1-1.fc17.src.rpm
 Uploading srpm: /home/maxam/rpmbuild/SRPMS/nortc-0.1-1.fc17.src.rpm
 [====================================] 100% 00:00:00 16.20 KiB 24.43 KiB/sec
 Created task: 1205719
 Task info: http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=1205719
 Watching tasks (this may be safely interrupted)...
 1205719 build (f17, nortc-0.1-1.fc17.src.rpm): free
 1205719 build (f17, nortc-0.1-1.fc17.src.rpm): free -> open (hsv-panda-3-v5tel)
 1205721 buildArch (nortc-0.1-1.fc17.src.rpm, noarch): free
 1205721 buildArch (nortc-0.1-1.fc17.src.rpm, noarch): free -> open (cdot-panda-7-2)
 1205721 buildArch (nortc-0.1-1.fc17.src.rpm, noarch): open (cdot-panda-7-2) -> closed
 0 free 1 open 1 done 0 failed
 1205719 build (f17, nortc-0.1-1.fc17.src.rpm): open (hsv-panda-3-v5tel) -> closed
 0 free 0 open 2 done 0 failed
 1205719 build (f17, nortc-0.1-1.fc17.src.rpm) completed successfully

——————————————————————————–

Finally, I managed to get my nortc script in an arm package format!

To test it in another Pi, I downloaded the finished rpm from the above link (in the snippet). Then I installed it and rebooted the Pi. When the Raspberry Pi came up it worked just like a charm! …err a finished package!

At some point soon, I will have to set up a repository where I can host the nortc package. For the time being, I have a wiki page with the minimum information available here.

Curing the memento(-ish)? effect of the Raspberry Pi

In the summer of 2012, I was working on creating images for the Raspberry Pi using Fedora. I was working as part of the Seneca Centre for Development of Open Technology (CDOT) team. Our goal was to create something that will be a good experience for an audience who is relatively new to Linux/Fedora. When it comes to creating a good experience, regardless of the OS, it comes down to paying attention to details. It was no different in our case. As part of that goal, one of the side projects that I worked on was to find a way to fix the time in the Raspberry Pi Fedora Remix 17, that was causing the OS to ask users to set up passwords over and over and won’t let them perform updates.

In traditional computers, usually at least one real time (CMOS) clock is built in. When an OS is booting in that computer, it uses that hardware to get the initial time and uses that till a network connection is established and then is uses some sort of network time protocol to readjust the clock . This is usually the case in most modern operating systems. As for the Raspberry Pi, it does not have a hardware clock! My initial thoughts were, I have to create a clock out of thin air that can produce time out of thin air! Fortunately it didn’t go that far! When all these talk about fixing the clock was going on, someone asked why there is a the need for that clock when there is network based time update. Well, if there is no clock present and the network is not up yet (early stage of booting, not connected to network), most (all?) OSes will default to the epoch time (1970). That is not a bad thing if it is booting up for the first time, but say it is happening over and over and the system time was updated through network time and files and databases got updated timestamps, then that becomes an issue. When a system boots up and checks time stamps in password files (password valid till date), if it is set too far away in the future (from the POV of that OS in epoch time), it will reject passwords and/or will ask to change it. (That was a consistent pattern in the early tests performed). Also, before even the login/password service starts there is possible extra file system check steps when the file system gets mounted. Then there are cases where package management utilities (i.e. yum) may not be able to sync with the server and error out because of the wrong timestamp. In our case, every time the Raspberry Pi booted up, it didn’t know the time and had no clue what time it was even before that. So, I had to find ways to give it some memento to remind it of the last time it was on!

I had two ways to approach that issue, one was to go and muck around with the kernel and make something fancy (a kernel module!) or I could just have a simple bash script activated reasonably early in the boot process to have the date/time module not switch to the epoch time. I chose the latter. When I was doing research on how to approach this issue, I found other scenarios where this clock fix thing could be useful, i.e. a broken RTC with dead battery can use a script such as this. So, I decided to make the script in a way that should allow it to run in other devices too. Along the way there were some challenges I had to address. Some of those were Fedora specific and some were generic. I had to decide how to make the OS recognize that this is not the first time it is booting and pick up the time from a specific location in the file system consistently. I went through gazillion trial and error process and after numerous heated, delightful and thought provoking discussions later it came down to have the time picked up based on file system creation or last mount or access time. There was some Fedora specific challenges with that approach which I have addressed later in the post. When it came the issue of how to decide which file system/partition to pick as the reference point, that was addressed via two different ways (get rootfs info from the mount and blkid). As for choosing the time, I had it pick the latest of all the options with the possible fallback to a recent date. After all that, the sudo code looked like this:

——————————————————————————–

Check if there is any RTC, if there is don’t fix the date/time.
Check if the date is set to something acceptable already, if it is don’t fix it.
Find all the possible root partitions.
Find the latest date from those root partitions.
Use that information to update the time.

——————————————————————————–

And the script looked like this:

—————————————————————————————————————————————————————-

#!/bin/bash
#
# Name: nortc: Time and date fix for devices with no RTC
# Desc: Fixes the time and date in devices with no realtime clocks,
# e.g., virtual machines, embedded systems, specific ARM devices, etc.
# By : Masihul ‘maxam’ Abed <maxamaxim@me.com>
# Date: Thu Aug 02 2012
#
# Copyright (C) 2012 Seneca Centre for Development of Open Technology
# Copyright (C) 2012 Masihul ‘maxam’ Abed <maxamaxim@me.com>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
# remove the double hashes to enable debugging, both at the top and the bottom of the script
##( # redirect log

##set -x # start debugging
##set -e # continue on error
##echo $$ # get process numder
function dateupdate ()
{
latest=$1
# update to latest only if it is actually the latest
if [ $(date +”%s”) -gt $latest ]
then
logger -p daemon.notice -t NORTC “Current date is ahead of nortc date. Date was not changed”
else
logger -p daemon.notice -t NORTC “Changing date”
date -s @”$latest” >/dev/null 2>&1
logger -p daemon.info -t NORTC “Date changed”
fi
}

function datefind ()
{
rootparts=$@
# assign a generic date if no usable root partition data
if [ ${#rootparts[@]} -eq 0 ]
then
latest=$(date +”%s” -d “Jul 01 2012″)
else
# if multiple partitions/names found then get their creation/update/access/mount dates
for part in ${rootparts[@]}
do
partdates=”$(dumpe2fs -h “$part” 2>&1 | egrep “^(Filesystem|Last).*(created|checked|time):”)”
IFS=$’\n’ dates=(${dates[@]} “$(echo “$partdates” | awk -F”^.*: +” ‘{print $2}’)”)
done

# get the latest of the creation/update/access/mount dates or assign a generic date
latest=$(date +”%s” -d “Jun 01 2012″)

for adate in ${dates[@]}
do
thedate=$(date +”%s” -d “$adate”)
latest=$((latest>thedate?latest:thedate))
done
fi

echo $latest
}
function rootfind ()
{
# try to get the root partition from mount, if that fails then try from bulk id
partlist=”$(cat /proc/mounts | grep ” / ” | cut -d” ” -f1 ; blkid | grep root | awk -F”:” ‘{print $1}’)”

for part in ${partlist[@]}
do
# keep the usable ones only
if dumpe2fs -h “$part” >/dev/null 2>&1
then
rootparts=(${rootparts[@]} “$part”)
fi
done

echo “${rootparts[@]}”
}

function datedetect ()
{
# log and exit if the year is 2012 or up
if [ $(date +”%Y”) -ge 2012 ]
then
logger -p daemon.info -t NORTC “Current year is greater than 2011. Date was not changed”
exit 0
fi
}
function rtcdetect ()
{
# log and exit if real time clock detected
if [ -L /dev/rtc -o -c /dev/rtc0 ]
then
logger -p daemon.info -t NORTC “RTC detected. Date was not changed”
exit 0
fi
}

# Run it

rtcdetect
datedetect
rootparts=”$(rootfind)”
latest=$(datefind “${rootparts[@]}”)
dateupdate $latest

##) > /tmp/nortc.log 2>&1 # unify log to a file

exit 0

—————————————————————————————————————————————————————-

With that basic script in hand, I now had to figure out a way to have it activated early in the system boot process. Since I was working on Fedora and it uses systemd, I had to learn how systemd invokes services and at which order they come in and so on. After struggling with it for two days, I managed to make a simple systemd unit that activated my service close to when I wanted at boot time. I was happy and went on to grab a cup of tea after turning on my RasPi and when I came back and checked, I didn’t get the result I was expecting. The unit file worked, but the script kept falling back to the default fallback time that I set. After checking the log that did not exist I enabled debugging and logging. Then I checked again and what i found was that Fedora mounts the rootfs twice, first very early in the boot process and then it does a remount. I was activating my script in between those two. After spending some time on google I came to the conclusion that had I went the kernel way, it would have been a better solution. I was looking for a solution and when I asked my teams Research Chair, he advised that in most cases the early mount usually is read-only, it was not implemented in the Fedora RasPi images. He had it addressed and fixed blazing fast and I tried with the suggested changes and the script worked as expected and the results were consistent afterwards. My final unit file looked like this:

——————————————————————————–

[Unit]
Description=time and date fix for devices with no RTC
DefaultDependencies=no
After=systemd-readahead-collect.service
Before=systemd-readahead-replay.service dev-hugepages.mount remount-rootfs.service fsck-root.service

[Service]
Type=oneshot
ExecStart=/usr/sbin/nortc
RemainAfterExit=no

[Install]
WantedBy=local-fs.target sysinit.target

——————————————————————————–

Now that I had a working script with mechanisms to activate it early in the boot process, I was ready to package it and test it in other RasPi devices. I have decided to do a separate blog post just for that and the link for that post is here.

favicon with png2ico: Where fun happens!!!

As part of a course, I was required to make a website using perl and cgi. I needed a favicon and I had a png image that I wanted to use as the icon. So, I was looking for a way to convert png files to ico files. As I use Fedora both in my MacBook Pro and the workstation for school stuff, I decided to use YUM to find something suitable. Fedora uses YUM as it’s Package Manager (app finder). Using YUM, I found an amazing image editing app called ImageMagicK. The best thing about that app –  a command line module! So I used  that and converted the png file to an ico file. The conversion process worked somewhat ok. For whatever reason the icon file that was being produced was darker than the original. At that point, I decided to use a GUI tool to fix it. But, then I asked myself, where is the fun in that?

As it happens, for my other course I was required to produce an RPM package using an existing tarball and that gave me an idea! Wouldn’t it be cool if I could find an app that could convert images the way I like and then package and use it to convert the image file for my other course? Well, that would be seriously cool! …(in a nerdy way)! So, I was in the market for an app that would let me convert images! I texted my good buddy Google and got a reply that, there may be an app fitting the exact description of what I wanted to achieve! It turned out to be a very simple utility that converts png to ico, aptly named png2ico! My only question was will it produce a better result than the other app? I had two choices. I could just download the tarball and install it or  I could google some more. So I asked myself again, where is the fun in that! I downloaded the tarball, extracted the files in a temp directory and started looking for the readmes and stuff. I read the err…. readme files (there was more than one!) and then looked at the content of the tarball, it had enough files to have it packaged both for Linux and Windows. There was a .cpp file in there so I figured it will use C++ to build. The included license file was GPLv2. Installation instruction was super simple! Just run make! So, I did just that… the result was not quite what I was expecting. It was looking for a Linux PNG Library package (libpng). YUM told me it was installed, then I remembered that to build it I will need the devel version of that library. So I installed the libpng-devel package and tried again. It complained again and failed, it was telling me that there was at least one deprecated header file (see log below), and I was also getting a permission denied error.

—————————————————————————–

g++ -W -Wall -O2 -finline-functions -g -o png2ico png2ico.cpp -lpng -lz -lm
In file included from /usr/lib/gcc/x86_64-redhat-linux/4.7.0/../../../../include/c++/4.7.0/ext/hash_map:61:0,
from png2ico.cpp:40:
/usr/lib/gcc/x86_64-redhat-linux/4.7.0/../../../../include/c++/4.7.0/backward/backward_warning.h:33:2: warning: #warning This file includes at least one deprecated or antiquated header which may be removed without further notice at a future date. Please use a non-deprecated interface with equivalent functionality instead. For a listing of replacement headers and interfaces, consult the file backward_warning.h. To disable this warning use -Wno-deprecated. [-Wcpp]
/bin/ld: cannot open output file png2ico: Permission denied
collect2: error: ld returned 1 exit status
make: *** [png2ico] Error 1

—————————————————————————–

Fixing the gcc warning was simple, I followed the error/warning suggestion and the warning went away. But I was still getting the permission denied error and at first I thought there was something wrong with the Makefile, may be it was trying to create files in a location that it didn’t have access to. Then I realized what I just thought and laughed at my own naivety…(newbie-ity?) and then I looked at all the file permissions and figured out the issue. I extracted the tarball as a root and then went on to make as a user. After a quick change of ownership I ran the make command and this time it worked! It built successfully!

Now to the next stage, packaging the software in a way  that can be installed in any Fedora box. Those who might be interested to learn more on how to package softwares for Fedora, can find information here. Fedora uses a software packaging process called RPM, I won’t go too much in details about how RPM works, but I will point to this link as I found it very helpful. To make an RPM package I needed to make a Spec file. A spec file is a way to automate the exact same process a user might go through to extract, make and install an app. To create a spec file one might start from scratch, use a template or copy an existing one from a different package  and modify it accordingly.

I chose the third option, I took an existing one and modified it. I listed ccache (g++) and libpng-devel as build requirements. When was running the make command I got a warning and had to modify the original Makefile.  Now that I am trying to automate that process, the preferred way to do that is to create a patch file. A patch file contains the changes made to an original file and  as the name suggests, patches those changes to the extracted tarball. One of my peers recently wrote a tutorial on how to make a patch file, I am linking it here for reference. The patch file I made for this project looked like this:

—————————————————————————–

– – – a/Makefile 2012-09-25 14:42:33.437998032 -0400
+++ b/Makefile 2012-09-25 14:42:04.076996435 -0400
@@ -1,9 +1,9 @@
-CPPFLAGS=-W -Wall -O2 -finline-functions
+CPPFLAGS=-Wno-deprecated -Wall -O2 -finline-functions
#CPPFLAGS=-O0 -W -Wall
DEBUG=-g

all: png2ico

png2ico: png2ico.cpp
g++ $(CPPFLAGS) $(DEBUG) -o $@ $< -lpng -lz -lm

—————————————————————————–

Then I went on to specify where to install the files, provisioning additional directories where it was necessary. I had the spec file ready and I wanted to make sure that it didn’t have any syntax error, so I ran it through rpmlint:

—————————————————————————–

[max@mbp png2ico]$ rpmlint -i ~/rpmbuild/SPECS/png2ico.spec
0 packages and 1 specfiles checked; 0 errors, 0 warnings.

—————————————————————————–

I was ready to build using the spec file, I copied the tarball and the patch file in the source directory and told rpm to build it ( rpmbuild -ba ~/rpmbuild/SPECS/png2ico.spec ), and I got an error. Rpmbuild didn’t like the directory name where the tarball was extracted:

—————————————————————————–

Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.ZBrGqk
+ umask 022
+ cd /home/maxam/rpmbuild/BUILD
+ LANG=C
+ export LANG
+ unset DISPLAY
+ cd /home/maxam/rpmbuild/BUILD
+ rm -rf png2ico-0.1
+ /usr/bin/gzip -dc /home/maxam/rpmbuild/SOURCES/png2ico-src-2002-12-08.tar.gz
+ /usr/bin/tar -xf –
+ STATUS=0
+ ‘[‘ 0 -ne 0 ‘]’
+ cd png2ico-0.1
/var/tmp/rpm-tmp.ZBrGqk: line 39: cd: png2ico-0.1: No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.ZBrGqk (%prep)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.ZBrGqk (%prep)

—————————————————————————–

So I added instructions in the setup section ( -n png2ico ) of the spec file so that rpmbuild would look into that directory for the extracted file, and it built a bit further than before but failed again. This time it was because I forgot to remove an architecture specific line from the spec file (I copied it from a package I created before).  After I removed that line, I tried again and this time it went without a hitch, I got my app!

I wanted to try it out, so I issued the following command:

rpm -Uvh ~/rpmbuild/RPMS/x86_64/png2ico-0.1-1.fc17.x86_64.rpm

It installed successfully. I read the man page to learn how to use it and issued the following command:

png2ico favicon.ico favicon.png

And the result was amazing! It created a better quality image compared to the ImageMagicK’s command line convert command! However, I have to say, ImageMagicK probably can create a similar (bettter?) image with some additional parameters. I probably could have done that on the first place. But, where is the fun in that!

For an inquisitive soul, this is what the final spec file looked like:

—————————————————————————–

Name: png2ico
Version: 0.1
Release: 1%{?dist}
Summary: png to icon converter
License: GPLv2
Group: Applications/Productivity
URL: http://www.winterdrache.de/freeware/png2ico/
Source0: http://www.winterdrache.de/freeware/png2ico/data/png2ico-src-2002-12-08.tar.gz
Patch0: png2ico-Makefile.patch
BuildRequires: ccache libpng-devel
Requires: libpng

%description
Converts PNG files to icon resource files. A program to create favicon.ico

%prep
%setup -q -n png2ico

%patch0 -p1

%build
make

%install
rm -rf %{buildroot}
mkdir -p %{buildroot}%{_bindir}/
cp png2ico %{buildroot}%{_bindir}/
mkdir -p %{buildroot}%{_mandir}/man1/
cp doc/png2ico.1 %{buildroot}%{_mandir}/man1/

%files
%defattr(0644,root,root,-)
%doc README README.unix VERSION LICENSE
%attr(0755,root,root) %{_bindir}/*
%{_mandir}/man1/*

%changelog
* Tue Sep 25 2012 Some User <user@example.com> – 0.1-1
– initial build for png2ico 0.1

—————————————————————————–

Building from source – Part 2: The Etherpad/Nodejs story

In part 1 of building from source I wrote about some general steps I followed to build an app/utility from source on a Fedora box. Today I will cover something that I found a bit more exciting. I will try to build Etherpad Lite from source. Etherpad Lite (el) is based on the same concept the original Etherpad was built on, to provide a shared editing/collaborating webspace. However, el uses quarter of the memory compared to the original Etherpad. It is possible because el uses nodejs as the backend which makes it highly scalable. As it takes less resources that allows opportunities to create more el instances/nodes.

The challenge now was to build both nodejs and el from source. In nodejs website instructions were provided on how to build from the source. I found additional information required to make it compatible with el from here. I started by installing the dependancies for nodejs from yum and then went through configure, make, make install process. Then I followed instructions here for Etherpad Lite and got it running. Etherpad Lite uses a file called settings.json and multiple instances could be created by duplicating that file and changing it. It runs from the same directory where it was extracted. I didn’t like that, I want my files/apps in specific locations. The second issue is that el/nodejs runs using web sockets. It is accessible via port 900X (X could be 1, 2, 3, 4…. depending on how many instances running), but there is no easy way to access those via apache. Then came the third issue, security and authentication, I looked up in the el github wiki and based on what different folks posted there, it is a nightmare to setup. The fourth issue I had was with the builtin file based DB. I wanted to run it using MySQL.

I want to look at each of those issues separately and figure how to make it work …at some point in the future. For now I got myself a el colab space that runs on ~256MB of RAM!

Make alive from source in Fedora (Building from Source – Part 1)

As part of a course that I am taking, I am required to build apps from already existing sources. I have decided to build it on a box running Fedora 17. I had to make sure that the OS was capable of building from sources, in Fedora context that means installing the Development tools and Libraries. It is really easy to do that part in Fedora just by typing the following command.

yum -y install @devel*

I am required to build two apps and one of those will have to be something that was crated by GNU Software Collection. I chose the package alive as my first one. I downloaded it from here. I extracted it in the /tmp directory. I switched to the extracted directory and read the README and the INSTALL files. The README file specified how to configure various options and features and the INSTALL file had directions on how to install keeping the generic configurations. One of the options I found interesting was the ability to compile for multiple architectures, considering it is a tiny package. But then again, it kinda make sense as this utility could be used in different servers. For the purpose of this course, I went with the generic approach. However, I did not do an actual install as it would have replaced my already installed alive package in Fedora. The commands I ran were:

./configure
make
# make install

I got an error saying that I was missing the guile package. I did a quick check in my system and realized that it was already installed. So I started googling and found out that in Fedora context I needed to install *-devel packages to meet the additional development criteria necessary to compile certain packages. In my case,  for alive that would be guile-devel. It didn’t get installed with the development tools and libraries. I ran the following command to meet the build requirement:

yum -y install guile-devel

Then I ran the configure and make again. This time it compiled without any error.

That is how I built/compiled a package from source. A detailed guide containing more information on this process can be found here.

Setting up telnet in Fedora Core 5!!!

So… I was trying to setup a Fedora Core 5 (FC5) Virtual Machine as part of a course…. I know right!…. FC5!!!! Fedora 17 readily available, why should someone try to install such an ancient build? Well as it happens it is part of a course that teaches basic security principles, and having an older build helps. The format of the course required weekly labs (following instructions in a wiki) to be completed.

So I managed to get the DVD iso image from Fedora Archive (took like guzzillion years to download!) and I kept most of the settings generic except I selected the Web Server Package Set. I rebooted the box and everything came up nicely except xinetd didn’t get installed. As part of the requirement for that lab it was necessary to install telnet (which required xinetd). At that point, I naively typed in the installation command and got an error since the repositories are long gone. I was aware that the DVD itself could be used as a local repository, but I didn’t know how to do that. I googled a bit and found instructions here. In my case the installation DVD got auto mounted as /media/Disk . I had to disable some repos as those were causing yum to exit with error. At this point all I had was one enabled repo with baseurl pointing to /media/Disk . I ran yum to install xinetd and that worked nicely. Then I went about installing on the rpm package for telnet.

This whole ordeal got me thinking… may be I should automate this and make life easier for some folks who may be inclined to install FC5 with telnet enabled, and I came up with this: (the DVD was already inside)

—————————————————————————–

cd
sed -i “s|^enabled=.*|enabled=0|” /etc/yum.repos.d/*.repo
cat > /etc/yum.repos.d/maxam.repo << “EOF”
[core]
name=Fedora Core $releasever – $basearch
#baseurl=http://download.fedora.redhat.com/pub/fedora/linux/core/$releasever/$basearch/os/
baseurl=file:///media/disk/
#mirrorlist=http://fedora.redhat.com/download/mirrors/fedora-core-$releasever
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora file:///etc/pki/rpm-gpg/RPM-GPG-KEY
EOF
yum clean all
yum -y install xinetd
wget ‘http://goo.gl/0E8So&#8217; -O telnet-server.rpm
rpm -Uvh telnet-server.rpm

—————————————————————————–

At this point I used the GUI to enable and start all the necessary services including xinetd and then selected telnet from On Demand Services.

Installing Fedora on a Mac in a Multi-boot Environment

The other day I decided that I will install a Linux OS in my MacBook Pro. I have Lion installed and decided to install Fedora in a dual boot scenario. At that time, my understanding was that Mac has some “magical tools” that allows people to install dual operating systems with relative ease. I quickly discovered that it only works if the choice of OS is Windows. I already tried Windows in a dual boot situation in a Mac and it ran better than any Windows I have ever ran in any PC, but I wanted to go the Fedora way this time!

So I googled a bit trying to figure out what other ways I can dual boot Linux on a Mac. I finally found a tool called reFit. I did some more googling and found out that it can co-exist with the native Mac EFI boot loader and can multi-boot! So I downloaded reFit and installed it. Then I went on to split the hard drive in a reasonable chunk and I used the built in disk utility tool.

I happened to have a Fedora 16 64-bit DVD handy and I popped it in and rebooted the Mac, reFit menu came up instead of the usual boot loader and I selected the DVD. Fedora installation menu came up and I went to clicking some buttons to do a generic install. I kept most of the settings as is, however I did select custom partitions and configured volume groups and logical volumes. I might have went a little overboard with the partitioning. This was what it looked like at the time of writing this post:

—————————————————————————–
[root@mbp ~]# parted /dev/sdb p
Model: ATA ST9250315ASG (scsi)
Disk /dev/sdb: 250GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags
1 20.5kB 210MB 210MB fat32 EFI System Partition boot
2 210MB 200GB 200GB hfs+ Customer
3 200GB 201GB 650MB hfs+ Recovery HD
4 201GB 201GB 2097kB bios_grub
10 201GB 201GB 522MB linux-swap(v1)
5 201GB 213GB 12.0GB lvm
6 213GB 225GB 12.0GB lvm
7 225GB 237GB 12.0GB lvm
8 237GB 249GB 12.0GB lvm
9 249GB 250GB 524MB ext4 ext4 boot
[root@mbp ~]# pvs
PV VG Fmt Attr PSize PFree
/dev/sdb5 aruba lvm2 a– 11.15g 0
/dev/sdb6 aruba lvm2 a– 11.15g 0
/dev/sdb7 aruba lvm2 a– 11.15g 0
/dev/sdb8 aruba lvm2 a– 11.15g 624.00m
[root@mbp ~]# vgs
VG #PV #LV #SN Attr VSize VFree
aruba 4 3 0 wz–n- 44.61g 624.00m
[root@mbp ~]# lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
f15_root aruba -wi-a- 20.00g
f16_root aruba -wi-ao 20.00g
home aruba -wi-ao 4.00g
[root@mbp ~]#
—————————————————————————–

Then I went back on clicking buttons and Fedora 16 was installed! I rebooted the Mac and by default reFit took me to Fedora (The default can be changed from Mac). Then I went on to customizing the shiny new Fedora that I installed. I customized it to my hearts content and then I decided to do an update. Updating was simple enough and after the update I rebooted the Mac. Fedora came up all nice and dandy! Well… not really… for whatever reason the wireless and sound stopped working! After hours and hours of googling I came across solutions for both issues. For the wireless issue I installed kmod-wl package and that did a reboot and I got wireless back. for the sound issue I installed alsamixer (command line tool). I ran it from a terminal, selected the right card by pressing Fn+F6 and unmuted all the channels by pressing ‘m’ and adjusted the volumes to something desirable. I saved the sound settings by typing ‘alsactl store’.

And that was it! This is my Mac’s smolt profile. Now at some point in the near future, I will try to do an upgrade to Fedora 17. But that is for another day!