Data driven investment and performance management in the livestock sector

Evidence-based decision-making is now axiomatic in many sectors and has become increasingly important in prioritising development in low- and middle-income countries. In the livestock development sector, there has been a lack of data on health and production required to establish an evidence base. Thus, much strategic and policy decision-making has been based on the more subjective grounds of opinion, expert or otherwise. However, there is now a trend towards a more data-driven approach for such decisions. The Centre for Supporting Evidence-Based Interventions in Livestock was established in Edinburgh by the Bill and Melinda Gates Foundation in 2016, to collate and publish livestock health and production data, lead a community of practice to harmonise livestock-data-related methodologies, and develop and monitor performance indicators for livestock investments.


Introduction
'With little or no knowledge regarding animal health, nutrition, production process, and reproduction procedures, and without a 41_2_20_Peters_preprint 2/25 scientific plan and program, achieving a desired economic profit is absolutely impossible' [1].
Evidence based decision making has become broadly accepted as being necessary for optimal decision making in many professional settings.
This has perhaps been most notable in the field of human medicine which began to emerge as a new paradigm for more rational clinical practice in the early 1990s [2] and has gained in wide global acceptance since then [3,4,5,6] and has become adopted in veterinary medicine [7,8,9,10] and many other professional and business sectors [11].
Evidence based medicine has been defined as the 'integration of best research evidence with clinical expertise and patient values' [12] and this principle has since been readily translated into other specific professional sectors.

The evolution of the livestock strategy in the Bill and Melinda Gates Foundation: a case study
A strategy for agriculture development in the smallholder sector of low and middle income countries (LMICs) was initiated in the Bill and Melinda Gates Foundation (BMGF) in 2006 and it soon became apparent that most smallholders rely both on livestock and crops for their livelihoods. However, there was a low base of understanding in the foundation of the livestock development priorities due to a distinct lack of data-based information on the needs of this sector [13].
Therefore a few initial investments were made in order to learn and to help inform future livestock investments; this process then resulted in the launch of a livestock initiative in 2012.
Seven countries in sub-Saharan Africa and two in South Asia were prioritised for dairy cattle (including water buffalo in South Asia), small ruminants and poultry, focusing on animal health and genetics, and the respective species' value chains. Fourteen infectious diseases considered to be causing the highest productivity losses were selected for investment. In genetics the focus was on identification and multiplication of appropriate genetic technologies applicable to smallholders, particularly in dairy and poultry. In prioritising and focusing these investments there was extensive consultation with 41_2_20_Peters_preprint 3/25 partners and consultants in order to gain expert advice on the most impactful direction to take. However, as the learning and experience accrued over those early years, it became ever more apparent that much of the input was based on experience and expert opinion rather than objective evidence, largely because of the extensive gaps in data from the field on these livestock health and productivity issues. The above pragmatic approach to decision making, rather than being evidence based can be illustrated by the example of how livestock diseases were initially prioritised for intervention by BMGF. The 14 priority diseases referred to above were selected on expert opinion and largely based on the report by Perry et al. [13]  infectious diseases was subsequently adopted as key priorities for GALVmed. These decisions were largely based on the advice given by experts on the basis of their experience and knowledge of the sector, 41_2_20_Peters_preprint 4/25 but with little or no direct input of solid data. Further diligence was carried out to support the prioritisation, but it was noted at the time that there was a distinct lack of substantial data to support those decisions e.g. product demand, feasibility of product development and deployment, or the likely impact of product usage. Equally the decision making process here could be criticised for lack of due consideration to endemic production disease and non-infectious health problems.
The above example of non-data-based decision making is not unique.
In fact the lack of data was becoming ever more apparent across the board in livestock development [14]. At the time of writing, BMGF now supports more than 60 programmes in the field of animal health, genetics, enabling systems, animal nutrition and offtake markets. As this investment portfolio expanded, the recognition of the need for better and reliable data became more urgent and acute for making informed management and investment decisions to optimise desired impact and to select future development programmes.

Decisions for what? Decisions by whom? Who are the decision makers?
Within the current scope of SEBI, decision makers can be considered at least at four levels: disease control programmes, or nutrition or some combination of these? Donors may be private or public organisations and in the latter case might be accountable to governments and therefore also integral to category 2 above.
Of course, none of the above categories of decision maker is homogeneous but rather a loose grouping of organisations with similar but individual needs. For example, a small scale producer with one or two cows will have different data needs from a commercial dairy with 200 cows. Thus data needs are very much dependent on the audience or 'customer' for the data.
One of SEBI's tasks was to establish, convene and lead the community of practice, Livestock Data for Decisions (LD4D). Its purpose is to drive better livestock decisions through improved data and analysis.
Whilst making significant progress in bringing the livestock data community together, to find solutions for the most pressing livestock data needs, a valid criticism of LD4D is that the focus has been on the data collectors and analysts, that is the supply side of the equation, without much consideration of the decision makers (the customers).
This consideration has now extended LD4D's focus to address the demand side by aiming to identify the specific needs of key decision makers.
Of course, the needs of the four categories of decision makers or customer groups above are quite different. For example farmers need information on individual animal performance, local markets, input prices, weather etc. and need the capability to evaluate the impact of management changes and interventions like disease prevention and breed improvement [15]. The other three sectors mentioned need different data sets but all consisting largely of aggregated livestock data.

What is meant by data and evidence?
'Data driven' has become a fashionable and possibly over-used expression and theme. However if it is for a strategic purpose then the specific problem, question or objective has to be posed first in order to inform what data is needed. So, while the answer may be data driven,

What is data?
Data in the present context is purely an individual or set of noncontextualised numbers or observations. In order for this to be translated into evidence there has to be an opinion or hypothesis or question to add context (see Table I, [16], https://oxfordreview.com/data-v-evidence/#summary). For example, a data point might be the average milk yield of cows in Tanzania. Then one might ask, has this changed in the last 20 years? For this one would need contextual information, evidence, and for this one would need a series of data collected over that period of time and one would need to analyse and interpret it to provide the evidence and then render it into a useful piece of knowledge that could be communicated to another party.
What is the overall objective of data processing?
Thus, in order to collect the right data, for the right metrics or indicators, a clear statement of the problem and/or objective(s) is needed. In the international development context, the usual device for this is to construct a Theory of Change (ToC) (Figure 1). Theory of change is an outcomes-based approach which applies critical thinking to the design, implementation and evaluation of initiatives and programmes intended to support contextual change [17]. A ToC is a tool to help describe the problem (objective) to be addressed, the changes that are to be made and the plan for activities to achieve that objective [18]. The key to this and effect, because it can be extremely difficult in these cases to confirm any attribution with certainty. It is also important here to differentiate the terms outputs, outcomes and impact [19].
In the private sector the analogous tool or device would normally be constructed and termed a business plan (BP), essentially setting out the business objectives and then subsequently filling in the necessary activities, processes and resources necessary to achieve them. An important component of both the ToC and BP is the use of metrics or indicators, based on real data, to monitor progress and to ensure that the plan is on track and being adhered to. Thus, data is needed to measure the outputs of the programme and to inform the future direction. For this the trends in the data are also important e.g. is national milk production going up or down? Is egg production going up or down?
How many more cattle are being vaccinated?

How we collect and use data to learn and adjust our course as necessary
Data gaps and flaws are well recognised in the international livestock development field [14]. While there are many sources of data on LMICs e.g. FAO, WOAH, ILRI, Centre for Agriculture and Bioscience International (more commonly known as CABI), World Bank Group, grey literature, peer reviewed publications, data is disparate and incomplete and, may even be extrapolated inappropriately from the industrialised world. Data on greenhouse gas emissions in LMICs are a case in point [20,21].
In their report 'Investing in the livestock sector: why good numbers matter', Pica-Ciamarra et al. [14] drew attention to the deficit in good quality data on LMIC livestock production and went on to describe the institutional necessity for the routine collection and analysis of data at 41_2_20_Peters_preprint 9/25 an official level but they pointed out that such routine data currently is of poor quality partly because of inadequate training of extension officers who are responsible for the process. Furthermore, routine data are collated on a complete enumeration basis which makes it extremely demanding and time consuming and thus the authors advocate a statistically based sampling approach which could make data collection more efficient and convenient. It should be noted that it is not only livestock data that is lacking in many LMICs. The 50 by 2030 project a multi-lateral initiative to produce, analyse and apply data to decision making across the whole agricultural sector has recently launched (https://www.50x2030.org) to attempt to address these issues on an international scale.
There are many publications on livestock health and productivity in LMICs in the peer reviewed and grey literature, but the evidence has rarely been systematically categorised and reviewed. A key objective of SEBI has been to systematically describe the available data and evidence on the high impact livestock diseases in SSA. Highly detailed academic studies of the epidemiology of livestock disease have been carried out in limited areas [22] but these can be prohibitively expensive and therefore unsustainable for the purpose of providing long-term routine evidence. Data collection and processing comes at a cost. It is important to have reliable data and it should be fit for purpose. The law of diminishing returns can be applied to the quality of data. Starting at a low base of quality, increased investment will lead to an increase in the quality and hence reliability of the data. However, as investment country. An example of the output from the systematic map is shown in Figure 2. The output has been developed into a visualisation format that can be interrogated for individual species, regions diseases etc. and is available online at https://www.livestockdata.org.
Since the performance of such reviews is highly laborious this process has now been automated such that this can now be achieved with machine learning protocols [25] and the intention is to extend this process to other species, languages, and countries, to add more structure to the body of data on the impact and prevalence of tropical livestock disease.

Monitoring and learning for continuous improvement
The BMGF team needs a constant flow of data in the form of 'dashboards' from funded programmes, particularly the farmer-facing ones so that their outputs can be monitored, data aggregated across programmes with the objective of being able track progress, identify opportunities to improve and to adjust operational directions if necessary. These and other data coming into SEBI are cleaned and processed as described in Figure 3. Standardising the format and structure of incoming data is a work in progress.
In addition, the team depends on these data to calculate the impact of investments and to inform future investment decisions. Calculations are also carried out by SEBI on the economic return for SSPs on their investments in animal health and genetics. Examples of these models are illustrated in Figures 4 and 5.

Investigation of mortality rates/risk
To investigate the use of a single metric e.g., mortality as a measure of a country's animal health status, the mortality rates were estimated (or more correctly, risk [26]) in three countries, Ethiopia, Tanzania and Nigeria. A further aim for this effort was to determine the most prevalent diseases causing mortality with a view to be identifying priorities for implementing control interventions. A variety of methods were used including government institutional data, farmer recall surveys, expert opinions, living standards measurement study surveys, literature review, laboratory data and real time diagnostic surveillance.
All these methods have recognised flaws [14]. Thus, it soon became apparent that the estimates within and between years were highly variable to the extent that the use of such a single measure was considered unachievable particularly given the huge gaps in systematic data collection. On further analysis since a high proportion of livestock mortalities in LMICs occur during the early weeks of life (SEBI, unpublished data) we have suggested the use of a young stock mortality metric as a more reliable indicator of overall animal health status of a country [26].

Data on national Veterinary Services
The World Organisation for Animal Health publishes regular reports on national Performance of Veterinary Services. A tool was developed to extract data from such reports so that it can be manipulated into a summary and analysable form at https://www.livestockdata.org/dataobject/performance-veterinary-services-pvs This interactive visualisation can be interrogated to focus in on specific countries, years, etc. and is illustrated in Figure 6.

Conclusions
The goal of data driven investment and performance management in the LMICs livestock sector is a work in progress. However, it is apparent that: