Research Tools Consultations

By Lachlan Musicman

As part of Research Bazaar we thought there would be space to provide generic IT consultation for attendees about good solutions to their problems. As someone with twenty years experience in IT, the most constant issues I’ve seen have been solutions that were inappropriate, hard to maintain for the end user, expensive, not fit for purpose, have (often expensive) vendor lock in, or a combination of all of these problems.

By no means do we ever think we could solve everyone’s problems, but non computer scientists are often unaware of the tools and systems available to them, or have no way to judge the value of the advice or quote that they are given.

To be fair, often people expect a lot more than they can afford, have no idea of the complexity or cost of implementation of their ideal solutions, or even the language with which to describe what they want.

This is something that Research Platforms are perfectly suited to provide, and I was lucky enough to be asked to be one of those consultants.

In the first ever running of this consultation, there was a queue almost immediately.

There was an atmospheric scientist that was looking for better open tools, in particular for presentations - “how can I replace PowerPoint”. I offered LibreOffice’s InPress originally, but that had been tried and wasn’t wanted - too much like PowerPoint. So I pointed her to Reveal.js a web based alternative that is as beautiful. Not something I’ve used a lot, but @stevage uses it for his mapping presentations that have toured the country and it seems a good fit.

Another, a bioscientist, was in the R stream, but was looking to expand their repertoire. It’s hard to go past Python and shell. They are both excellent tools in their own right, and my knowledge of large scale supercomputing (thanks Pip@bio!) is that a rudimentary in shell is almost mandatory. Shell can be a long slog, but it is worth it. The concept of stringing a lot of small, single purpose, utilities together can be hard to understand at first, and the admitted arcane interface is intimidating, but it is powerful and most can get away with a small understanding and a dozen commands.

Python is much easier to get up to speed with, is as ubiquitous as shell, and with a hour or two of research and testing, incredibly powerful scripts

  • small single or simple purpose programs - can save many weeks of pain. I can’t recommend How to think like a computer scientist enough. It’s a free book, written by a high school teacher for his IT class, so while the first couple of chapters can seem easy going, by the end of the dozen or so chapters readers will feel comfortable doing powerful things quickly and simply. That it is a language that reads easily helps no end - just saying the code out loud can give you a good understanding of what it is trying to do. Another plus for Python is it’s applicability and extensibility - there are many libraries already written to do what you need, and is as useful for a sysadmin as it is a bioinformatician, a web developer and a digital humanist.

IMG_2150

I was surprised how much of the advice needed was web related. I had three people ask about websites for their projects. This is a multidimensional problem.

First and foremost, most researchers and teams, especially those seeking grant money, can over think their needs and wants before consulting a web developer. In the last five years, the number of people that have asked me for a site that includes the phrase “kind of like Facebook” and then followed it up with “we have no money” is reaching into the dozens. Here is the hard truth - Facebook has been built over a decade with billions of dollars, and includes work by psychologists, some of the planet’s brightest developers and millions, then billions, of users.

Find a consultant to talk about what you need first. They will help narrow your needs into at most four or five deliverables that will make it easier to work out what you need. For as little as $1000 you can have a pretty decent - not gorgeous, blue collar - data entry system that will collect your data, your data will be well modelled and there will be an API that will provide you with the scaffolding required to hang other applications off it. A supercharged spreadsheet, for want of a better analogy.

The speed of development will provide you with a good feedback mechanism so quick and easy tweaks can happen without breaking the fundamentals. This has two distinct positive outcomes - the developer(s) will have a greater understanding of your needs, and your users will have greater agency of the built software.

From that base you will be able to build further applications on top. Building websites is an iterative process (thanks Flanders!), so the smaller each part or deliverable is, the easier it is to make them work together well. Rome wasn’t built in a day, as they say.

The next most important thing to remember, and this goes to the heart of Research Platform’s raison d'etre, is that open is always better than closed. When a company wants to charge you $2 million for a site, it must be understood that there is a high likelihood - in the realm of 99% - that you will be locked into that provider for life. If they don’t perform to expectations, you are left with sunk costs that can’t be recouped if the project isn’t open. Proprietary code that is unavailable, often poorly commented and incomprehensible if available, and highly specialised is a lock in that no one can escape. When you use open tools and platforms - like Django, Ruby on Rails or Drupal, you have a plethora of companies available to you should you decide you need a new provider, or even want to just employ a small team of developers yourself.

Finally, there are often pre-made systems - like Omeka, CKAN, Zooniverse, Tumblr, MapBox/TileMill - that are free and will get you 90% of the way to where you need to be. They are open, often free, and will reduce your costs significantly.

From the other perspective, it is important to remember that web development is done by teams of people that are highly experienced and that doesn’t necessarily come cheap. Design and user experience, in particular, are fiscally expensive, back end and development support will be tax your time and mental understanding of what you want to achieve. You will be asked to think slowly and clearly, and if you are on a budget, you may need to think hard about “must haves” versus “desirables”. You will be asked to think hard about your problems in a way that can be translated into what can be achieved. This is an ongoing process, and often involves a distillation of big ideas into smaller, achievable ones. The consultants aren’t dismissing your greater goals or dreams - they are thinking about the possible, the plausible and the achievable. Patience is required from both sides. But if you can save yourself a bunch of headaches and $1.5 million, it’s probably worth it.

Research Platforms offers a weekly meetup - the Hacky Hour, Thursdays, 3pm at the University of Melbourne’s Tsubu, where you can get advice about your project. If you think your project needs a web application, come down and ask us. We can’t build it for you, but we can help you think more clearly about what you need, and what it might cost. And depending on what you need, we may be able to provide you with either a quick and dirty solution or a new experimental platform that we are testing.

As a developer, we love and respect your hard work and domains of knowledge in which you outstrip us - often we are fascinated and explosively excited to help and work with your projects. We are there to help your project in our domain of expertise. Come visit us.