When I started using computers about 30 years ago, the floppy disk was the standard of personal data storage. I actually started out using the 5.25″ disk so the 3.5″ disk with it’s hard case and a little bit more space was a welcome improvement at the time.
We’ve come a long way in the last three decades and now we have flash drives that can store tens of thousands of times as much data as the old 1.44 MB disk. Although smaller sizes are still available, the smallest flash drive you’re likely to see now can carry 8 GB of data which would have been enough to backup my first hard drive a few hundred times over.
While file sizes have gotten much bigger, that’s still a lot of data to carry around, especially if some of it is of a personal nature. That has its own risks as I found out first hand a couple weeks ago.
One of CiviCRM’s strengths is the ability to add custom fields to hold specific information about your contacts. One thing it doesn’t offer (yet) is a calculated field type that will present the results of calculations of other fields. While calculated fields are generally discouraged in relational database design, they are sometimes necessary within a user interface. One suggested method is to add custom code hooks within CiviCRM’s PHP code but as a database guy, I decided on a back-end solution.
On a recent project, I setup CiviCRM and CiviCase to enable a local organization to better manage its client database. The old database had been developed in Borland Paradox and was quickly becoming unusable. As a free and open source solution, CiviCRM turned out to be just the solution needed to accommodate this non-profit.
While CiviCRM itself does allow for the import of data through its API interface, I found that the contacts importer rejected a good portion of this data. The data had been entered by many volunteers into a database which did not impose many constraints on what data was required or how it could be entered. As with any case history, this data was essential to working with clients. With the volume of data involved, manual re-entry was not an option.
Fortunately, while CiviCRM’s data model is sophisticated, it’s relatively easy to decipher and I was eventually able to import the bulk of the data into CiviCase using a series of MySQL queries and MySQL Workbench.
I’ve been fascinated by the idea of an earth after humans ever since I read some of Alan Weisman’s book The World Without Us. The idea of humans “destroying the planet” has always been a necessary bit of hyperbole; we are far more likely to destroy ourselves as a species by making the planet uninhabitable. After we’re gone, it will eventually recover and go on just fine without us. Still, the idea of “saving the planet” communicates the scale of the problem and the potential loss in a way that many people have an easier time grasping.
Aside from the environmental issues, videos like this show just how strong the forces of time and nature are. So many of the structures that we take for granted crumble and disappear relatively fast without constant maintenance, from the giant buildings and bridges to the subways and power systems. Much of our influence on nature, including specialized breeding of animals and cultivation of plants, would eventually be undone without humans around to maintain it. Ultimately, nothing we do is really permanent.
This becomes more immediate when brought down to the individual level. We can hope that humanity will never really destroy itself, although we’ve only been flirting with the possibility for about 100 years so we haven’t had a lot of time to really explore the possibilities. What is certain is that each of us will move on as individuals at some point. Very few of us leave behind names that will be spoken centuries from now but we all affect the lives and environments that we touch which then affect others. We define ourselves, in large part, by what we leave behind, either when moving on to a new job or city or in our final moments. We cannot truly move on without caring for what we leave behind. The willingness to stop every so often, turn away from the noise and distractions of everyday life and reflect on our real effects on the people and world around us is an exercise in the personal integrity that ultimately determines what kind of legacy we will leave.
One of my current projects involves migrating a large amount of data away from an old custom Borland Paradox application into a new CiviCRM system. As with too many quickly-constructed apps, this old Paradox database wasn’t especially well designed and, among other the other challenges in salvaging the data, there were no restrictions on how dates could be entered. This means that, in multiple fields within each of the 20,000+ database records, I might see any of the following:
While CiviCRM does have a utility for importing data from CSV and other SQL tables, it was having quite a time with this collection and many of the dates were being mishandled. That’s if the records weren’t rejected entirely for other reasons. Data migration doesn’t often happen with just a few settings adjustments and a click of the Import button.
Years ago, when I was making do with the limited computer equipment that I could afford, I never dreamed that I would one day be able to login to a website, plug in a few specs about the machine I wanted and then, a few minutes later, log into that machine remotely and run whatever programs I needed to. Yet, that’s exactly what today’s cloud computing resources enable me to do.
In my last post on coding basics, I talked about turning an algorithm into code and used Euclid’s Algorithm as an example of programming a sequence of steps. There’s an even simpler type of algorithm that I want to look at this time. A formula, such as the one for converting Fahrenheit to Celsius, is also a series of steps that must be performed in order to achieve the needed result. These calculations can be represented within C# or other programming languages to include the formula as part of a larger program.
If you’re doing a one-time conversion from Fahrenheit to Celsius, you’re probably going to pull out your pocket calculator or find a good conversion website and enter the necessary numbers. You can do that because either of those resources has the necessary interface to get the necessary input from you. If you’re including this calculation in a program, you need to design your own interface, process that user input from it and return the result to the user in some way. In the case of the pocket calculator or website, other programmers pondered these issues at some point and now it’s your turn.
I’ve been working on a number of different projects lately, from I.T. networking to a book on MySQL, so I haven’t had as much reason to break out the programming tools as I used to. If not used regularly, programming skills can get a little rusty or even disappear like old friends from your college days that you lost contact with.
So, I’ve decided to delve back into the subject and update my status as a .NET programmer. The first step is a quick review of the C# language. One resource that I can recommend for this is The C# Programming Yellow Book by Rob Miles. It’s just $0.99 for the Kindle version on Amazon.com and you can even get a free PDF version from the author’s website.
In previous articles, I’ve talked about the importance of finding the right algorithm, or series of steps to follow, when coding a solution. Efficiency in terms of the amount of memory used and the amount of time taken by the operation are key factors for the program. Sometimes an appropriate algorithm is already available and in wide use and it’s just up to the programmer to turn it into code. There’s always the option of running to StackOverflow and grabbing some code but that does nothing to further your talent.