Chronophage's Blog

I recently had a client who has a client that does arbitrary PCI scans. Unfortunately, the service that does the PCI scans does not understand Ubuntu version numbers. So I had the joy of wrangling all of the CVE data from Ubuntu and compare it to their actual releases.

Being of a lazy temperament, I obviously didn’t want to do this by hand. Being of a curious temperament, I wondered if I could do this all from the command line.

First thing I had to do was to take their PDF and convert it to a format where I could extract the CVE strings. I’ve previously used pdftohtml, it works well enough. Now I needed to grab the actual CVE strings themselves without grabbing the rest of the line. The strings are in the format of CVE-{year}-{4 digit number}

SED to the rescue:

# cat report_pdf.html | sed -n 's/.*\(CVE-[0-9]...-[0-9]...\).*/\1/p' > cve_strings.txt

Now I needed to grab all of the CVE reports from Ubuntu and save them locally:

Enter Awk:

# awk -F\- '{system(curl" " "http://people.canonical.com/~ubuntu-security/cve""/"$2"/"$1"-"$2"-"$3".html") }' cve_strings.txt

Next, I converted all of .html files to text. Again, another tool I’ve used in the past html2text. However, one thing that gave me fits was the weird text format it outputs to facilitate underlines and bold caracters all using ascii characters.

Underline text looked like this: _U_n_d_e_r_l_i_n_e_d T_e_x_t_
Bold text looked like this: BBoolldd TTeexxtt

less and cat interpreted properly, but now grep (grrr…) So I had to use the -nobs option.

Here’s a little for loop to create the text files from the HTML files:

# for i in `ls`; do cat $i | html2text -nobs > `echo $i | sed 's/.html/.txt/g'`; done

Finally, I had to parse the CVE text files. Now, each vulnerability could effect one or more packages. So I wanted to generate a report that had the CVE number, and then the status of 10.04 under each package.

There are easy was to do this, but nested for loops on the command line is much, much more fun:

# for i in `ls CVE-20*.txt`; do echo \\n`echo $i | sed 's/.txt//g'`; IFS=$(echo -en "\n\b"); for k in `cat -s $i | grep Source:\ `;do echo $k; cat $i | grep -a4 $k | sed "s/_/\ /g" | grep 10.04 ; echo \\n; done; done > report.txt

As you can see, the version string for Ubuntu was inconsistent, it was either Ubuntu_10.04_LTS and Ubuntu 10.04 LTS. Hence the extra sed command.

Was this the most efficient way? Well, probably not. But it took me a lot less time than scripting the process outright.

Wee!

So, I decided I needed a bit of a landing page for web.chronophage.net. I wanted to show certain active statistics, but I didn’t like the format, or information leakage that mod_status showed.

So I did some googling, and found this: http://www.phpclasses.org/package/3613-PHP-Retrieve-and-parse-Apache-server-status.html

Unfortunately, the examples were set to just show the stats of one server, and I had four.

Well, an array, for loop and some php rejiggering, and viola! Active server stats!

Now I can see all the worms attempting to XSS my websites.

Wee!

So, I changed mail, splitting mail.chronophage.net into a 3 host mail cluster, and a shell server.

Unfortunately, I have shell users who were using mail.chronophage.net as their moniker for pop/imap. This also broke their ability to log in to the antispam area and change their spam settings. I have fixed both of these issues.

Dovecot has a passwd-file facility, which uses passwd formatted files to do authentication. I simply added such a file to my mail cluster, with the shell username and {CRYPT} hashes (out of master.passwd) and added a proxy=y statement and a host=(shell server’s ip)

So now, when they log in, it’ll authenticate them, and then pass the authentication over to the shell server’s POP3 or IMAP server. Since they authenticate locally first, SASL works for sending mail.

This gives them one point for external clients, and still lets them log in via the shell server, or use procmail (via a .forward) if they so choose.

Unfortunately, this means that I have to manually keep password in sync in two places. But my users don’t change their passwords often, and there aren’t too many of them

So, for posterity

mail.chronophage.net (POP3, IMAP, SMTP)
https://mail.chronophage.net (webmail via roundcube)
https://mail.chronophage.net/squirrelmail (webmail via squirrelmail)
https://mail.chronophage.net/antispam (for antispam services)
gaia.chronophage.net (shell server)

Working on a post for work. Part 1 is pending edits and approval. Here’s a chart I’ve made for Part 2:

*UPDATE* Post approved: http://blogs.iphouse.net/2011/11/10/infrastructure-and-other-games/

So I did a little bit of load testing on my new web cluster.

Not bad for not having a real load balancer…

(continue reading…)

I’ve been waiting, and working.

I’ve been waiting for my work to release a its new product. I’ve been waiting, politely, for my boss to blog about it. I’ve been waiting to show off this new product.

I’ve been working on provisioning, and working with customers on beta testing the new product. I’ve been working on templates, and auto install media, to make everyone’s life easier. I’ve been working on documentation for customers.

I’ve been waiting for, and working on, a VMware vCloud Director based product known as vmForge VDC.

This is cool stuff!

(continue reading…)

Well, in honor of IPv6 day, I decided to spurn people along. I created an IPv6 validation badge of my own, and a few other things to prod people to get their providers to switch.

I’ve also updated http://ipv6.chronophage.net with a new look. It also shows you a video that warns you of the real and present dangers of not switching to IPv6, or celebrates your IPv6 achievement with an INTERNET classic

Making the pages was fun, especially using php to spit out the various dynamic graphics. In one spot I use a validator lifted from here On the main page i simply have this SetEnvIfNoCase REMOTE_ADDR "^[0-9a-f:]+$" IPV6_USER=1 in my apache config, and check for that variable in my shtml index. I use a php filter elsewhere. This is to test various ways of IPv6 validation. If you see an error, please comment.

Making dynamic graphics was straight out of the php documentation. That language has a function for everything!

No one will accuse me of being a website designer, I hope it’s ugly enough for you.

Anyways, have fun! Happy (pending) IPv6 day!

So…

It’s been awhile.

Recently, I’ve decided to make sure that all of my servers were IPv6 addressable. This was made infinitely easier by working at a forward thinking ISP. So a quick email to our network admin and bam! IPv6 routed to my vlan!
Now, what to do with it?

(continue reading…)

(..sigh) There’s a lot I like about RedHat. I like the fact that they’re worth more than a billion dollars as an OpenSource company. I like how they’ve been in the Linux game for a long time, and keep finding new ways to innovate and expand. I like how they have a competitive, yet accessible application stack, that runs on a wide variety of hardware. Their partner program, and resources are outstanding, especially when compared to other software vendors. I like a lot about RedHat. I just don’t like their operating system.

(continue reading…)

Man, I’ve been busy… I’ve recently been promoted to being a System’s Administrator! This has forced me to program a few BASH scripts. This one is for a customer that wants a managed system, but wants to be able to add websites at will. It’s pretty simple, and relies on an existing example.com template. Why example.com? Because I follow RFCs dammit! The template allows me to adjust the Apache specific settings, without recoding the script.  A little REGEX here, and a pipe to sed there, some error checking, formatting and a dash of some SUDO magic and voila! The customer doesn’t need to email/call me to add a website.  Sure beats paying for a crappy control panel

(continue reading…)