Recently we’ve needed some functions for working with dates and business days, so I looked for some help on this.

I found a good post here: http://archives.postgresql.org/pgsql-sql/2009-11/msg00038.php

In the process, I made a few changes, including the use of Postgres 8.4 ‘with’ syntax, which I find really useful in making things understandable. Obviously, this needs Postgres 8.4 to work.

Here are the results, in case anyone out there is looking for similar functions (any comments / suggestions / improvements welcome):

Holiday Table
First, create a table containing all the holidays that aren’t working days:

CREATE TABLE holidays (
   id serial,
   description character varying (50) not null,
   holiday date not NULL
)
WITHOUT OIDS
TABLESPACE pg_default;

insert into holidays (description, holiday) values ('Christmas Day', '2009-12-25');
insert into holidays (description, holiday) values ('Boxing Day', '2009-12-28');
insert into holidays (description, holiday) values ('New Years Day', '2010-01-01');
insert into holidays (description, holiday) values ('Good Friday', '2010-04-02');
insert into holidays (description, holiday) values ('Easter Monday', '2010-04-05');

Add Working Days to a Date
Now, create a function to add working days to a given date:
NOTE: If you input a non-working day and add 0 days to it, you will get no result!

CREATE OR REPLACE FUNCTION addbusinessdays(date, integer)
  RETURNS date AS
$BODY$
with alldates as (
	SELECT i,
	-- Check for negatives
	$1 + (i * case when $2 < 0 then -1 else 1 end) AS date
	-- NOTE we add 5 and x 2, to make sure the sequence has enough in it to cope with weekends/hols that get omitted later
	FROM generate_series(0,(abs($2) + 5)*2) i
),
days as (
	select i, date, extract('dow' from date) as dow
	from alldates
),
businessdays as (
	select i, date, d.dow from days d
	left join holidays h on d.date=h.holiday
	where h.holiday is null
	and d.dow between 1 and 5
	order by i
)
--select count(*) from businessdays where date between '2010-04-01' and '2010-04-07';
-- now to add biz days to a date
select date from businessdays where
        case when $2 > 0 then date >=$1 when $2 < 0 then date <=$1 else date =$1 end
	limit 1
	offset abs($2)
$BODY$
  LANGUAGE 'sql' VOLATILE
  COST 100;
ALTER FUNCTION addbusinessdays(date, integer) OWNER TO postgres;

select * from addbusinessdays('2009-12-24', 1)

Finally to get the number of working days between two dates:

CREATE OR REPLACE FUNCTION countbusinessdays(date, date)
  RETURNS bigint AS
$BODY$
with alldates as (
	SELECT i, $1 + i AS date, extract ('dow'  from $1::date + i) AS dow
	FROM generate_series(0,$2-$1) i
),
businessdays as (
	select date, dow from alldates dt
	left join holidays h on dt.date=h.holiday
	where h.holiday is null
	and dow between 1 and 5
	order by date
)
select greatest(0,count(*)-1) from businessdays where date between $1 and $2;

$BODY$
  LANGUAGE 'sql' VOLATILE
  COST 100;
ALTER FUNCTION countbusinessdays(date, date) OWNER TO postgres;

select * from countbusinessdays('2009-12-24','2009-12-29');

That’s it! Hope it’s useful to someone.
Bob

Having said that however, we are responsible citizens  and would like to contribute back where we can. Generally, that is either in code contributions or by paying for the software that we use. Code contributions are difficult, we don’t have a lot of time to spend on making changes to the software that we use. We also use far more than would be practical for us to be across. So we pick one or two systems that we really need more from and contribute to those.

For the rest, we would love to be able to pay reasonable amounts of money for them. The issue is that invariably the pricing structure is either free or enterprise grade (i.e. >= $10k per CPU per year) style pricing. This includes support, escalations etc. The trouble is that this leaves us with no option to really pay anything.  Personally I would have thought there would be thousands more companies like us than there are major enterprises willing to pay. I would like to see products costed at $1k per year for access to the product (via RPM) and security updates. I don’t need the support (not at that price anyway).

The products that do provide that level of pricing, I pay for, the rest, I use for free. While I feel guilty about that, until the business models of the companies providing these systems change, there is not much else I can do.

Cheers,

David

We are putting together our backup solution – we use Bacula to manage our backups which is an excellent network aware solution that seems rock solid for us. We needed to compile our own RPMs from the source, but this was a simple matter of running:

# rpmbuild --rebuild --define "build_centos5 1" --define "build_postgresql 1" \
                     --define 'build_gconsole 1'--define 'build_bat 1' \
                     bacula-<version>.src.rpm

which we then imported into our yum repository. The configuration for this was pretty complicated the first time we set it up, but with the excellent HOWTOs on the site was not too much pain. We then had to decide where to store all the data. Tape drives are expensive, and slow and was not something we really wanted to do if we didn’t have to, so we built our own SAN. A SAN (Storage Attached Network) is a device which looks like a normal disk device on the computer, but in fact sends all the data across the network to a storage device.

Building a SAN is as simple as installing correct package and configuring it up:

# yum install scsi-target-utils
# chkconfig tgtd on
# service tgtd start
# /usr/sbin/tgtadm --lld iscsi --op new  --mode target --tid 1 \
                   -T iqn.2009-01.example.com.san1:storage.backup
# /usr/sbin/tgtadm --lld iscsi --op new  --mode logicalunit --tid 1 --lun 1 \
                   -b /dev/VolGroup00/Backup
# /usr/sbin/tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL
# echo >> /etc/rc.local <<EOF
/usr/sbin/tgtadm --lld iscsi --op new  --mode target --tid 1 \
                 -T iqn.2009-01.example.com.san1:storage.backup
/usr/sbin/tgtadm --lld iscsi --op new  --mode logicalunit --tid 1 --lun 1 \
                 -b /dev/VolGroup00/Backup
/usr/sbin/tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL
EOF

and you are done. Then you need to bind to that storage on the server that requires access to it. This is as simple as:

# yum install iscsi-initiator-utils
# echo "InitiatorName=iqn.2008-11.example.com.`hostname`" \
  > /etc/iscsi/initiatorname.iscsi
# iscsiadm -m discovery -t st -p <san address>
# iscsiadm --mode node --targetname  iqn.2009-01.example.com.san1:storage.backup \
           --portal <san address>:3260 --login
# mount /dev/<new device> /mnt

You now have a working SAN hooked up to the machine that requires it. Couple this with drbd and you have a highly available storage setup which will run as fast as your network (Gig-E is obviously recommended for this). Using a distributed filesystem (such as GFS2) should allow many machines to access the same data simultaneously. However, we have not done this as yet.

Database

We have two machines, both with Postgres 8.1 installed on them (the latest version provided as part of CentOS 5.2). While apparently 8.3 can work in active/active mode, we decided to stick with 8.1 to reduce dependency hell with everything else on the machines and work with DRBD. Setup is incredibly simple – we created an /etc/drbd.conf file which had:

global {
  usage-count no;
}

common {
  protocol C;
}

resource r0 {
  device    /dev/drbd1;
  disk      /dev/LVMGroup/LVMVolume;
  meta-disk internal;

  on <node1> {
    address   <ip address1>:7789;
  }
  on <node2> {
    address   <ip address2>:7789;
  }
}

on both nodes and ran :

# drbdadm create-md r0  <-- on both nodes
# service drbd start    <-- on both nodes
# drbdadm -- --overwrite-data-of-peer primary r0  <-- on the primary node

This started DRBD and allowed the primary node to sync to the secondary. For more details about this (and heartbeat configuration below), have a look at this excellent CentOS HOWTO. Then we needed to configure heartbeat to manage the automatic failover for us. Create /etc/ha.d/ha.cf on both nodes to contain:

keepalive 2
deadtime 30
warntime 10
initdead 120
bcast   eth0
node    <node1 name as per uname -n>
node    <node2 name as per uname -n>
crm yes

The /etc/ha.d/authkeys on both nodes should contain:

auth 1
1 sha1 <passwd>

This will then result in a working heartbeat. Start the heartbeat service on both nodes and ywait a few minutes and the command crm_mon will show you a running cluster:

[root@<node1> ha.d]# crm_mon
Defaulting to one-shot mode
You need to have curses available at compile time to enable console mode

============
Last updated: Tue Dec 16 14:29:03 2008
Current DC: <node2> (33b76ea8-7368-442f-aef3-26916c567166)
2 Nodes configured.
0 Resources configured.
============

Node: <node2> (33b76ea8-7368-442f-aef3-26916c567166): online
Node: <node1> (bbccba14-0f40-4b1c-bc5d-8c03d9435a37): online

Then run hb_gui to configure the resources for heartbeat. The file that this gui configures is in /var/lib/heartbeat/crm and is defined via XML. While I would prefer to configure it manually, I haven’t worked out how to do that yet and the hb_gui tool is very easy to use.

Using that tool, you can create a resource group for the clustered services (click on Resources and Plus and select Group). Then within that group you need to configure 4 resources, a virtual IP address that can be used to communicate with the the primary node, a filesystem resource for the DRBD filesystem, drbddisk resource and a resource for postgres. To take each in turn:

1. IP Address – click on plus and select Native, change the resource name to ip_<groupname>, select the group it should belong to, then select IPaddr from the list and click on Add Parameter. Then enter ip and a virtual ip address for the cluster. Add another parameter nic and select the interface for this to be configured against (i.e. eth0). Then click on OK.
2. drbddisk resource – Same procedure, but this time select drbddisk  instead of ipaddr and select Add Parameter. Then enter 1 and the name of the drbd resource created (r0 in our case).
3. filesystem – Same again, but select Filesystem and add the following parameters:
   1. device, /dev/drbd1 (in this example)
   2. directory, /var/lib/pgsql (for postgres)
   3. type, ext3 (or the filesystem you have created on it)
4. postgres – Lastly add a postgres resource with no parameters.

Start the resource group and in a few minutes everything should be started on one of the nodes. To switch over the the other run service heartbeat stop on one and everything will migrate to the other. Good luck, you should now have an active/passive cluster for your database. This worked for us. Your mileage may vary, but any issues feel free to leave a comment and we’ll update this HOWTO.

Web

Creating the clustering for the Web was similarly easy. We kept the 2 web machines as they were with Apache and Tomcat running on both and instead clustered the load balancers initially in active/passive (until we can work out the active/active settings) in much the same way. The key difference was that for these machines we ran the load balancing software (HAProxy) on both all the time and the cluster just looked after the IP address. That way nothing was slowed down if the primary load balancer failed while waiting for services to start.

Cheers,

David

Installing Samba was a little complicated, there are any number of HOWTOs out there to follow and while initially there was a big learning curve, it got easier quickly. Zimbra contains a very explicit HOWTO for integration between Samba and LDAP so while that took a little thought it also worked without too much heartache. It also meant that integration with the various systems were a little easier to debug as well.

Once we had followed the HOWTOs it was a simple matter of migrating the workstations to the new domain. Samba v3 is modelled on the pre Active Directory technology, so the whole process is a little more manual than with the latest Windows technology. However, v4 of Samba promises to include all that functionality in which will make things a lot less manual.

It did mean that we now could reuse 3 Domain Controllers (and not have to purchase Client Access Licences for all the users to be able to connect to the domain controllers with) and save the subsequent licencing costs. Creation of a new user account now occurs within Zimbra, but the login scripts have to be manually set on the Samba server. Telling Samba to obey the pam restrictions on the server ensured that home directories would be created automatically however.

The two biggest issues we encountered:

1. Until a recent version of Zimbra, if someone changed their password within Zimbra it would not be reflected on the Windows machines. This is now rectified, via an extension, but did require user education.
2. Samba inexplicably removed a feature by which we could filter the ldap searches on active users, without replacing it with anything similar. As a result I have not been able to find away to disable ex employees from being able to log on just by setting their account status in Zimbra. This is just plain stupid. Instead I have to write a script to check the Zimbra status and apply the same to the Samba statuses. Really, really irritating. If we weren’t saving so much money on the Windows Servers, I would seriously reconsider going to samba again. The workaround is such that if a user gets disabled inadvertently (i.e. they type their password in incorrectly and get locked out) then I need to re-enable them twice.

Leaving the negatives aside, Microsoft makes you buy Windows Server and then Client Access Licences to be able to use the server. Then more licences for anyone wanting to log in to the server (and I swear you have to be a rocket scientist to understand the Terminal Services licencing documentation). It all adds up. Samba is more stable than Windows, fast and works well. Other than the inability to hook into LDAP correctly which I did create a workaround for… Lets hope v4 works better in that respect.

Cheers,

David

As a result we had a look around at ticketing systems that we could use to manage the customer correspondance. Because we were dealing with end customers, a traditional CRM system didn’t really meet our needs. We do not have a long history with most of our customers – instead dealing with product questions, returns etc. We wanted something more lightweight, but still capable of being able to track customer history if necessary and provide management reporting etc. Any solution we look for now also has to integrate with LDAP. Zimbra uses LDAP internally and provides extensions to allow for single sign on. This will allow us to manage user accounts centrally.

Having used Zimbra, we now had some Linux expertise in house (more on this later) and decided to trial Request Tracker. This is a Web based tool that receives emails sent to specific addresses and delivers them to Queues set up within the system. It also has an FAQ Management system (RTFM) which integrates with Request Tracker allowing our staff to formulate standard responses to common questions. This saves time and effort. You can define arbitrary queues, and groups of users, permissions assigned to a user/group/queue combination. It is an incredibly powerful tool and because it is written in perl easily extensible (if you know perl). It supports a multitude of databases ensuring that it is likely to support a database  you are already using.

Of course, as with a lot of Open Source systems, this is also the major problem as well. Because *everything* is configurable, it means you have to configure everything. They options are mind numbing. Once it is set up, you should not have to play with the permissions too much more, but if you do, be ready to start scratching your head for a while. In addition to this, because the system is so easily extensible, some functionality is not easily available unless you write some perl code. For instance, it makes sense to define a single automated response, rather than trying to create 20-30 of them for each queue, but if you do that, there is no obvious way of stopping a particular queue from using the global one. Unless you write some perl code. There are also some extremely odd artifacts in the user interface – for instance, when replying to a ticket, rather than having the recipients checked and allowing you to uncheck people who shouldn’t receive the reply, instead everyone is unchecked and you have to explicitly click on the ones who should not receive the replies. A usability expert would have a field day with that one.

While RT integrates partially with LDAP, it will not take the group membership from LDAP, so you still have to manage every user in two places. This is painful. The other integration issue we have yet to address is to tie in customer tickets with our internally developed system to manage customer profiles etc. Ideally these would be linked. We plan on revisiting this when we look at ERP implementation.

So is RT a good solution? Well it is certainly very powerful and if you can live with the geekyness (and have perl expertise in house), I would recommend it. For us, I would drop it in a minute if there was a more user friendly solution out there. Any suggestions?

Cheers,

David

Our basic strategy is to migrate as much as possible into Web based applications. This reduces desktop support costs as well as hardware costs (eventually – when we have no desktop applications required any more). As such we went looking for a Web based email solution. We had a look at a variety of solutions and eventually settled on Zimbra. This looked to be extremely full featured with a Web interface that was exceedingly close to that of Outlook (2003) which the users were already used to.

Running through our due diligence on the software we even discovered that Microsoft themselves could find little to criticise (see: Microsoft’s curious infatuation with Zimbra). With Zimbra being Open Source, we downloaded it and trialled it with some of our users to find it worked, very very well. It has built in anti-spam which immediately cut down on a huge percentage of the Spam received, while providing a usable interface for most of the users.

One user didn’t like the fact that messages were marked as read in the preview pane (this was subsequently fixed by Zimbra in their v5 release) and for users that had to download attachments regularly, it was slightly less well integrated with the Desktop than Outlook was. However, the increase in efficiency as a result of the Spam solution more than made up for that. With the v5 release, the cross mailbox conversation view is astonishingly good and everyone who has used it has commented on this. It is far better functionality than we have see in any Desktop app. An additional benefit is that the same interface is available when travelling as within the Office (without requiring a VPN to be set up).

While we started off with the Open Source version, we migrated over to the Network Edition after 6 months – mainly because we had migrated critical emails from the Desktop PST files that Outlook had used to ensure that we had backed up email on the server. This resulted in a much larger Email store than anticipated and with the Open Source edition, we only had offline backups which started taking in excess of an hour to run. Migration to the Network edition allows us to have online backups which makes life much easier. It also cost far less than upgrading to the full version of Exchange.

The upgrades for Zimbra are excellent – it is being actively worked upon and each new release brings us functionality that we find useful. The latest version (as of October 2008), v5.10 apparently provides automatic translations courtesy of Yahoo – which for our Customer Service team is of major interest. We shall be testing this shortly.

I would like to find a downside to our transition to Zimbra, but the experience has been great. We can highly recommend it. While there was some initial resistance to migrating away from a desktop email client, a year later everyone is using the Web interface smoothly.

Cheers,

David

The primary software used in many businesses is of course, Microsoft Office. On the original machines we were mostly licenced for MS Office, but different versions with different subsets of software available. Everyone used Word and Excel, some used Powerpoint and everyone used Outlook for email. Unfortunately, whoever had purchased the licences for Office had only purchased OEM licences which meant that as we replaced hardware we would have to re-purchase software we had already bought. Ouch. In addition to this, we were on Office 2003 and would have needed to buy Office 2007.

As a result, we reviewed our options and decided to look into Open Office. I had only previously used Microsoft Office and entered into the review with a degree of skepticism. My preconception was that it would be pretty good, but not quite as good as MS Office. I assumed we would be sticking with MS Office as a result. However, the review was most interesting; it turns out that Open Office has a different approach to document management than Word – it really pushes you into using a style based solution. Having used Framemaker back in the early 90′s this was a blast from the past.

The advantage of a style based solution when you are in a company with more than a roomful of people is that it makes it very easy to create document templates which enforce a specific style and merging of multiple documents (for things like a weekly report) becomes very easy to create a consistent look and feel between multiple authors. Word obviously supports styles, but I have always found with Word that it tends to splinter unless everyone is incredibly disciplined about keeping the same style. In a small company that tends not to be the case.

In addition to this, the look and feel for Open Office was much closer to Office 2003 than Office 2007 was. Reading reports on the Internet there was a degree of consensus that the effort involved in training would be smaller for Open Office than the new Microsoft suite.

So, having trialed the software with the more technically savvy of the company, we made the plunge and migrated to Open Office, saving us the cost of 50 or so licences of Microsoft Office. A saving in the thousands of dollars.

One month after the migration, I had a total of 12 support requests from the users asking how features worked. Most of these were already covered in our migration documentation, so we pointed them to that in the first instance. As part of the migration we didn’t mass convert any documents, but left it to the users to save in the new file format as they opened them (to minimise costs). This saved huge amounts of disk space as the ODF file format appears to be far more space efficient than the MS format.

We have now been excusively using Open Office for 18 months and generally we are very happy with it. It does however come with some issues that were unexpected and have required some effort:

1. If someone else is editing a document, it does not tell you who it is. Instead it comes up read only. When you have people in multiple offices, this becomes an issue.
2. We have two sites with a wireless link between them. Every so often Open Office will lose the connection for some reason and be unable to save the document. Mostly this is worked around by saving to a local disk and copying back to the network. However, sometimes the Open Office format seems to get very broken and the only option is to save as an MS format and then re-open it and re-save the file again. Nasty.
3. There are some areas of incompatibility which don’t appear to be high on the radar to be fixed. One example: Calc does not understand the Excel A:A syntax for referring to an entire column. While these are not the biggest of issues, it is disappointing to see that the only competitor to MS Office which proclaims its compatibility is unable or unwilling to fix these problems. Luckily using Google to find solutions is pretty easy.

Other than that it has been smooth sailing. We plan on fixing the first two issues by moving off file servers and migrating to a document management system, then editing will all be local and we should not see those issues again. If you are just using file servers however I would strongly recommend you test large documents on the network before jumping into Open Office. If you can live with that however it is really very good, and you can’t beat the cost.

The last aspect of MS Office we needed to look at then was Email. More on this later.

Cheers,

David

As we were using Exchange, Outlook was the primary Mail client with users suffering from the inevitable .pst corruptions that occur when mail folders exceed 2 Gig. As all Customer Support was run via email directly to people’s personal email addresses there was a lot of email flowing around and a high likelihood of corruption.

The business also relied on a Visual Basic app installed on everyone’s desktop which interacted with the customer database.

Looking at the Visual Basic application there was no way that could be replaced in a short period of time, so we decided to replace the machines as they hit their EOL with new leased Windows Desktops with on site support. This would turn out to be pretty much cost neutral as it cut out the need for local desktop support for the hardware. The (large) percentage of the Desktop Administrator’s time could be redirected to more productive uses.

Given these machines would be consistent we were hoping we could achieve economies of scale with their management. In the event, we didn’t want to pay Microsoft for a second copy of Windows licences (we already received OEM copies with the desktop lease) so we did not end up, as originally planned, with a single master image which could be deployed across the network.

We still wanted to install the software remotely however. A search around the Internet came up with OCS Inventory which looked to do what we needed. We installed this and while rather geeky, does indeed allow us to distribute software to all the Desktops remotely. It suffers from the same problem that so much Open Source software does – it has not been designed with usability in mind. Given this is an Administration tool, that wasn’t a major concern, but it is something that would be good to improve on in the future. At time of writing, we had used it to install the initial software, but no upgrades as yet. Deleting software may also be a issue with this, however we will deal with that when we come to it.

Are we happy with this? Well it is not a bad solution. Given a choice of running thick clients and running thin clients with Terminal Services or Citrix we chose the thick clients. The setup, licencing and maintenance of the thin solution looked to be too hard given the huge amount of other tasks we had to deal with. Given that the VB app was a key requirement it was a good decision we feel, even if the desktop hardware was substantially more expensive.

It does leave us with desktop support issues – application support is still somewhat of an issue, however, when the lease is up (in 2 years or so), we plan to revisit this decision. More on this later…

Cheers,

David

For the software we did have, we were only partially licenced and we obviously needed to resolve that situation as quickly as possible. The servers were outsourced at an outrageous cost to a third party whose service kept having issues. The firewalls were not spectacularly secure either  – there was no VPN functionality set up and the owners who travelled a lot were using Remote Desktop to access files on the network which was not at all secure.

We had to decide how to fix all that and not spend more money than we had to play with. At a personal level I loath dealing with Software Vendors – the licencing terms require a mammoth amount of time/brainpower to work out what you are and aren’t allowed to do and in some cases, the licencing terms feel more like a way to gouge more money out of you than a reflection of their value. Then come the audits…

If car manufacturers tried to sell you a “cheap” car and then charge you extra to have a passenger, I wonder how quickly they would go out of business? Why do people put up with this from Software Vendors?

At a professional level, compliance costs to ensure that licences that are used are legal and that we are not breaking the terms of use can get expensive and in a small business you don’t have someone to manage that for you. Having read the first 8 pages of Microsoft’s Terminal Services Licencing documentation and still being unclear as to what we were actually allowed to use was just painful.

Lastly, I feel these days that Vendors are actively trying to lock you into their solutions and when running a business that is a major hidden cost to consider when selecting a solution. I would prefer to spend a little more on initial implementation if that saves running costs down the track than I would to buy something more cheaply up front which ends up costing much more in the log run.

So we decided to look at (free) Open solutions that would meet our needs. In some instances we went with proprietary solutions where we couldn’t find a good alternative. We’ll be sharing that with you over the coming months. We’ll be focussing on two main areas – Systems Administration and Development. We will discuss the approach to both we took and the platforms we decided to work with.

Cheers,

David