VizMyGrant: Visualising the NHMRC funding landscape and winning health hack

by Scott Ritchie

For the last two years software developers, user experience designers, data analysts, visualisers, and scientists have gathered together for HealthHack: a weekend data hack and competition to solve problems facing medical researchers. This year’s HealthHack took place on the weekend of the 24th–26th of October in Melbourne and Sydney.

Basically a bunch of smart people show up on Friday night, hear about the problems the problem owners (a.k.a. researchers) are facing. Questions are asked, and teams are formed. Everyone goes home for the night, and returns bright and early on Saturday morning for a dumpling and coffee fuelled weekend of hacking. Each team’s solution is presented on the Sunday afternoon, and judged based on both the quality of the idea and its execution: What approach was taken? Could it be generalised? Did it generate excitement? Did the team deliver in the weekend? To what extent was the problem solved? 

This year I attended HealthHack and joined Team VizMyGrant, who worked together with Dr Marguerite (Maggie) Evans-Galea from the Murdoch Children’s Research Institute to create visualisations of the Past and Present NHMRC funding outcomes. We ultimately went on to win the Melbourne HealthHack event for our solution. 

So who are Team VizMyGrant, and how did we make this possible? Read on to find out!

The Problem
Every year medical researchers across Australia eagerly await the results of the NHMRC funding decisions. After submitting an application in March and rebutting the reviewers’ comments mid-year, it will not be until October that they find out if they have received funds or not. It’s an exciting and terrifying time for everyone: Will they receive funding this year? Or will they have to join someone else’s lab? Or even contemplate moving overseas? 

Although the NHMRC makes their data publicly available, you have to pore through spreadsheets, PDFs, and Word documents if you want to go any deeper than the summary statistics the NHMRC provides. A couple of years ago, as Founding Chair of the Australian Early-Mid Career Researcher Forum with the Australian Academy of Science, Maggie spent many hours doing just that: examining Excel and PDF files, identifying imbalances in funding across gender and career stage, and overlaying this with published graduate and workforce data.

image

Having presented this information to policy makers in the course of positive discussions, Maggie was naturally keen to know whether the situation had changed since. However, rather than manually poring through documents to construct this information, she was hoping that a group of talented hackers could automate the process.

The goal at HealthHack was to build a tool that could generate user-friendly graphics from the publicly available NHMRC data to examine these trends for the latest funding rounds, and examine how they had changed across time.

The Team
Enter our crack team of VizMyGrant. In total eight of us decided to work on the problem, including myself and Maggie. On friday night we had been asked to designate our “super powers”: the skill sets we had at our disposal. Many of us held multiple super powers. Between the eight of us, we had  5 data visualisation experts, 4 data analysis experts, 3 software developers, 2 scientists, and 1 user experience designer. 

image

Pictured from left to right: Aaron McAleese, Sean Fleming, Iulian Stefanica, Irith Williams, Dr. Maggie Evans-Galea, Scott Ritchie. Not pictured: Fred Michna, and James Walter.

Dr Marguerite (Maggie) Evans-Galea (@MVEG001)
The problem owner. She is a molecular biologist, author, educator, and mentor. Her day-to-day research involves developing therapies and biomarkers for neurodegenerative disease. She is an advocate for science and research, and particularly the research workforce. She has spent many hours examining issues of funding, career structure, peer review, and gender equity in the context of science policy.

Scott Ritchie (@sritchie73)
That’s me! A Ph.D. student at The University of Melbourne. His day-to-day research focuses on the integration of large, complex datasets using network models and statistics with the hope of gaining new insight into the molecular changes underlying the manifestation of disease. He has a background in Computer Science and Bioinformatics. He also has a passion for data visualisation and science communication. 

Irith Williams (@IrithWilliams)
A user experience designer, researcher, and veteran hackathon participant. Her super-powers include: wrangling problems with sticky-notes, butchers paper and sharpies; video editing, and project management. She manages the LinkedIn group ’Designing for Health in Australia’. 

Iulian Stefanica (@stefanicai)
An experienced software developer and budding data visualiser. He primarily develops in Java, doing both frontend and backend development. He is a veteran hackathon participant, having participated in this year’s GovHack on the same team as Irith Williams. He is the founder of ResearchYo.

Sean Fleming (@a_sean_fleming)
An experienced software developer who specialises in databases and web development. His expertise lies in .Net, SQL, but he also dabbles in chef, nodejs, mongodb, and a few other technologies.

Aaron McAleese (@kiwintessential)
A freelance analytics consultant providing data development, statistical modelling, insights, visualisations and marketing strategy for charities, retailers and online services.

Fred Michna (@fredmichna)
A freelance consultant who specialises in maps, geographic data analysis, citizen sensors, spatial survey design, and qualitative data analysis. He is conducting a case study for the development of an autism service directory and map in a smart health information portal, and is looking for other interesting health information cases to work on. He also works in ecology; he is involved in a long-term project to set up a private field research station in the rainforests of far northern Queensland. He is looking to reach out to interested researchers and artists.

James Walter (@jamespwalter)
A logistics analyst for Australia Post. He specialises in data visualisation, geographic data, mapping, and general analytics. Outside of work he specialises in marketing analysis and logistics for Not-For-Profit organisations.

Overcoming our challenges
The biggest challenge our team faced was getting and cleaning the data. Without a clean dataset to work from, no visualisations or tools could be created. 

Irith was absolutely instrumental for getting everyone on the same page and making sure we had time to produce a final product. We followed her “double diamond process”: we had set time limits for each task, after which we had to come back and work out our next steps.

We spent the first hour of Saturday morning independently gathering data: trying to pull down as much NHMRC funding data from wherever we could find it. We reconvened to share our results: Where were the data? What were the variables we found? What did we think was going to be important? The resulting wall followed:

We found that although data collection was relatively straight-forward, many of the key variables Maggie was interested in were missing: successful applications were missing gender, career stage, and age of the applicant. We were also missing comprehensive information for rejected applications.

Together, Sean, Irith and I worked on arriving at a pre-determined format for the cleaned data with our combined knowledge of databases, data structures, and requirements solicitation. The hope was that once we knew what the cleaned data would look like, people could start working on the solution even if that data didn’t exist yet.

James and Fred worked on some of the inference problems. James found a handy R package for inferring gender from an applicant’s first name, and Fred obtained latitude and longitude coordinates for each of the institutions that received funding. Maggie created rulesets for inferring career stage and grant type from the available data.

It took us most of the Saturday to arrive at a cleaned dataset just for 2014. Reconvening before dinner we were left with some tough calls to make about what could be achieved in the remaining 24 hours. We made the decision to just work with the 2014 data: we wanted to have a solution to work from before considering the previous years’ data.

What worked well
The final stage of Irith’s double diamond process was to all diverge to work on different solutions. Each of us had our own ideas about what we wanted to create: whether that be an interactive tool, or a particular set of visualisations. Working on separate solutions gave us several advantages:

  • No overhead: we didn’t have to spend time agreeing on the solution, divvying up tasks, or stepping on each others toes.
  • Redundancy: there was no single point of failure. If any one solution proved to be infeasible, we had the rest of the solutions to fall back on.
  • It allowed each of us to play to our own strengths. None of us had to learn a new programming language or work on an unfamiliar technology stack.

This was only possible because we had narrowed down the scope of the problem. We didn’t need to work together on one solution to produce useful results for Maggie.

Another thing our team did well was social media engagement. We knew from the beginning that the problem and the solution had an incredibly broad impact, affecting researchers across Australia. As each new tool that was deployed, or trend came to light we communicated them to twitter via our own personal accounts, Maggie’s specially created @VizMyGrant handle, and via the Open Knowledge Foundation (who organised HealthHack).

The Solution
In the end all of our individual solutions proved to be both viable and useful. Between us we deployed four interactive online tools, created a video, and generated visualisation reports. Sean collated all the interactive tools onto a single landing page.

Irith and Maggie worked on the team video: storyboarding and storytelling, to create a compelling 3 minute video to show to the judges. The video is embedded earlier in the article, but can be found here on youtube.

I built an interactive tool using R and Shiny that dynamically generates plots based on arbitrary combinations of axes, filters, and data groupings. User experience consultation with Maggie had revealed that some visualisations would mainly be used as print-outs, so I utilised photocopy and colourblind friendly colour palettes. The tool can be found here.

Sean created a tool for visually comparing the relative number of grants awarded across groups using nested circle plots. It was built using the D3.js javascript library, and can be found here. 

James created several interactive visualisations using Tableau that show the distribution of funding across genders and career stages, institutions, and states. One interesting trend that he identified is that the Northern Territory punches well above its weight, receiving by far the most NHMRC funding per capita, with Victoria following as a distant second. The visualisations can be found here.

Fred took a literal interpretation of “the funding lanscape” and created a topographical heatmap of the funding outcomes in 2014, using the institution geo-locations to create funding mountains. The East Coast fared particularly well, with Melbourne being the proverbial Everest of NHMRC funding. The visualisation can be found here. 

With the rest of us working on solutions, Aaron and Iulian decided to work on the past data, successfully cleaning the data from 1996 onwards. Although we didn’t have time to incorporate it into our tools, they applied their data analysis skills to create key visualisations from the time series data. Visualisations for funding outcomes since 1996 can be found here.

So has the funding landscape changed?
The short answer is no. At the early career researcher level, more women than men were funded, but received on average 90 cents to the dollar. Mid career saw a massive bottleneck for both genders, and then at the senior levels the money and grants overwhelmingly went to male chief investigators. Further, looking at data since the year 2000 indicates that the overall gender balance in funding has changed very little over the last 15 years.

image

The Future of VizMyGrant
VizMyGrant has the potential to be useful in three different ways for the research sector: for individuals, research organisations and at a national level (funding bodies and policy-makers). Breaking down complex data into easy-to-interpret figures can be very helpful in short communications and meetings. With appropriate resources and support, it would also be possible to extend the capabilities of VizMyGrant to incorporate research workforce modelling. This would need to be done in collaborations with the national science funding bodies (NHMRC and the Australian Research Council). Discussions are currently underway with the NHMRC to determine how VizMyGrant could potentially assist their ongoing work.

——

Want to learn more about #HealthHack? Read our event roundup here, the organiser Maia’s musing here, and Tim Hildred’s (from Open Source) here.  And check out the official storify.

Want to plan an event like #HealthHack and join the Open Science movement? Get into contact with the Open Knowledge Foundation. Ping them on Twitter or check their website.

Special thanks to Maggie Evans-Galea and Irith Williams for their suggestions and copy editing