Difference between revisions of "35 Management Information System"
m |
(No difference)
|
Revision as of 12:58, 30 September 2021
Chapter 34 'Monitoring and Evaluation' describes the role of monitoring and evaluation (M&E) in assisting strategic planning and management. Chapter 35 aims to complement the earlier chapter by describing the background leading to the emergence of a WMG tracker in 2016/17 as a response to:
- a growing capacity of WMGs in four key areas: economic development, water management, organisational management and capacity development;
- recognising the value of a dashboard to monitor the adoption of Blue Gold norms by individual polders;
- the value to the TA team of hands-on health checks conducted with specific WMGs; and
- the positive response from water management organisations to participatory monitoring and eventually to self-assessment.
The final sections of this chapter account for the development of a polder dashboard to provide an overview of the progress of individual polders towards the goals of the Blue Gold Program, and how 'health checks' of WMGs gave insights into progress, achievements and challenges, and allowed project resources to be targeted more effectively and efficiently. And finally, a discussion about arrangements for the continuation of self-assessments by WMGs after the project.
Background[edit | edit source]
The original main purpose of monitoring and evaluation (M&E) in Blue Gold was to monitor and evaluate progress and achievements against key performance indicators of the program at output and outcome levels, with the aim of enhancing evidence to use in planning and management of the program. Information on, and the progress of different activities was collected by technical assistance (TA) sub-teams - then known as ‘components’ - who analysed the data and presented progress and shortcomings against the targets set in the Annual Work Plan - which in turn was based on the project logframe.
From the outset, data related to the different TA activities were collected – at output level as well as at outcome level. This data was then compiled and analysed, and reports were prepared:
- TR06 household surveys for P22, P30, P43/2D and P43/2F (31 Mar 2014)
- TN07 FFS Cycle 4 benchmark and end data: Mar-Sep 2015 (23 Dec 2015)
- TR14 Phase I Socio-Economic Baseline Survey Report (28 Dec 2015)
- Master Files[Notes 1] with geography, present situation, economic sectors and potential value chains for P2, P22, P26, P29, P30, P31-part, P43/1A, P43/2A, P43/2B, P43/2D, P43/2E and P43/2F (various dates from Jun 2014 to Sep 2018).
Figure 35.1 shows how data was collected, stored and distributed before the introduction of a digital management information system (MIS). The data were collected on paper and entered into spreadsheets, and were then analysed by specific individuals. Access to information by other project stakeholders was only then available through project reports, as processed data .
Establishing a WMG Tracker[edit | edit source]
In August 2016, as part of a refocusing of the scope of the monitoring, reflection and learning team, an activities plan[1] was prepared and discussed with the 2016 Annual Review Mission (ARM). This included a prototype WMG tracker which had been developed in September 2016. During its visit in October 2016, the ARM recommended that the strengthening of WMOs should “be supported with an adequate MIS system”, and that the TA’s prototype WMG tracker should be converted to a dashboard type MIS with target setting that is open access, with quarterly reports from the MIS provided to all stakeholders. (Recommendations 2.3 and 4.4)
A provisional format for data collection for the WMG Tracker was prepared in January 2017 and given to polder teams for a trial. WMG data was collected by TA polder-level staff, Community Development Facilitators (CDFs). Steps to overcome a difficulty with accurate data entry were agreed in March/April 2017 and the revised format was finalized in May 2017. A first report on the WMG Tracker[2] for the period up to June 2017 was issued in November 2017. WMG tracker data continued to be collected by CDFs on a quarterly basis for each of the Blue Gold WMGs.
Management Information System (MIS)[edit | edit source]
A vast amount of data is collected from the WMG Tracker and from field surveys and the participatory monitoring system. A rationalisation of the systems for data collection and management aimed to collect and store data from different sources at one location (a central server), which would allow better analysis and data presentation to aid decisions for project implementation. The main objectives of the Blue Gold MIS were therefore to:
- Collect and store information collected by the program in one place/server
- Standardise the information collected by the project;
- Present the data in standard reports that can be shared with program partners; and,
- Make the MIS accessible so that interested users can obtain and analyse data.
The ultimate aim of providing a consistent set of data for analysis, data presentation and reporting, was to facilitate informed, timely and evidence-based project decisions.
Figure 35.2 shows the proposed M&E data management system, in which data collected through electronic device are stored in a single location, the server where data can be analysed and presented in an MIS dashboard. In addition, data can be downloaded from the server in MS Excel for further analysis as may be required – all stakeholders have real time access to data and the dashboard without having to wait for a report to be generated.
MIS Design and Development[edit | edit source]
In March 2017, Blue Gold contracted mPower Social Enterprises of Dhaka, to develop an MIS platform. Their specialist team included system design and database analysts, mobile and web app developers and quality assurance engineers. During the inception phase for mPower's project, the technical data requirements of Blue Gold stakeholders were mapped and the project activity chain was agreed. Proposed features of the MIS platform included customizing Open Data Kit (ODK) software so that data could be collected through android devices, and uploaded into a custom-made, cloud-based data collection system with a visual reporting dashboard at the front-end.
The MIS system was ready in October 2017, and output results from the WMG tracker were first presented to BWDB during a workshop on 25th October 2017. Training in the design and use of the database was then provided to staff of BWDB and DAE both in Dhaka and in field offices. The MIS system was used from June 2017 to June 2019, when the collection of data was discontinued because of the expectation (in June 2019) that there would be staffing reductions in preparation for closure of the program at end-June 2020.
| Blue Gold’s MIS system is a mobile, web-based solution with built-in forms for the collection and combination of historical data (from previous data sets), baseline data, WMG tracker data, participatory monitoring as well as outcome and impact assessment data. Data is entered through an android-based mobile application using standardised forms which – after completion – can be saved locally and reviewed and then submitted to the server.
Users are able to view and export raw data and review reports. Those with administrative user rights are able to create, assign, edit and build new functions into the system. Data required for reporting can be exported imported into the M&E database from the MIS, providing a quick and simple way to share the data without disrupting the proven MIS processes. All M&E databases and reports are available in the MIS. The MIS can also be linked to a GIS-enabled solution so that information can be depicted graphically on a cadastral map, with interrogation possible at division, District, Upazila, Union and WMG level. |
MIS Results Reporting[edit | edit source]
Reports on the gathered data were available through a dashboard as well as in various working papers.
One series of working papers, WP9 (see below and in the File Library), presented an analysis of information of data collected through the WMG Tracker. They provided evidence of growing WMG capacity in four key areas: economic development, water management, organizational management and capacity development.
| 22 November 2017 | WP 9A | WMG Tracker Report-June 2017 |
| 15 April 2018 | WP 9B | WMG Tracker Report-December 2017 |
| 20 June 2018 | WP 9C | WMG Tracker Report-March 2018 |
| 27 August 2018 | WP 9D | WMG Tracker Report-June 2018 |
| 26 January 2019 | WP 9E | WMG Tracker Report-September 2018 |
| 20 February 2019 | WP 9F | WMG Tracker Report-December 2018 |
| 16 May 2019 | WP 9G | WMG Tracker Report-March 2019 |
| 30 October 2019 | WP 9H | WMG Tracker Report-June 2019 |
BWDB’s Project Coordinating Director (PCD) used the dashboard, analysed and cross-checked data, and monitored the activities and progress of WMGs. He used information from the dashboard in preparing articles for publication in newspapers, making presentations, and shared information on WMOs with other BWDB officials.
WMG Tracker Closure[edit | edit source]
Plans to discontinue the WMG tracker were made during 2019, as part of a gradual phasing out of TA staff in expectation of project closure at end-December 2020, and with reductions in TA polder-level staffing (the staff who collected the raw data) planned at end-December 2019, and in the absence of a stakeholder willing to take over its management. The final report was issued in October 2019 based on information collected from April to June 2019.
Polder Dashboard[edit | edit source]
To provide an overview of the progress of individual polders towards the goals of the Blue Gold Program, ten proxy indicators were identified from project monitoring tools to represent the four broad thematic areas towards empowering WMGs:
- overall achievement
- organisation
- water management, and
- economic development
For each of the ten indicators, targets were set for achievement by end-June 2020 but without defining the modality, thus freeing polder teams to select implementation arrangements which best suited the local environment, in other words, providing ‘a clear destination’ but with some flexibility as to how to get there. Figure 35.3 provides an example of a dashboard summarising results as of June 2019. A short note accompanied the dashboard to identify actions required to achieve the targets, and to identify the achievements for each polder and to recommend corrective actions (for example, see the note dated September 2019).
The polder dashboard was in use from December 2018 to September 2019 but discontinued when information for the WMG tracker - from which some of the proxy indicators were derived - ceased to be collected from end-September 2019.
Polder "Health Checks"[edit | edit source]
In addition to the quantitative data obtained on polder and WMG performance, qualitative polder “health checks” have been carried out by multidisciplinary teams of TA staff (see example output in Figure 35.4). These health checks were intended as a third-eye, qualitative assessment, which in combination with quantitative project data, provides insight into progress, achievements and challenges in specific polders, and help the team to target project resources more effectively and efficiently.
| Topic | Snap shot of discussion | Outcome & Remark |
|---|---|---|
| Goal | Members think WMG is for achieving something jointly. They gave emphasis to water management for development. | This is an organisation based on a temple. It is a close-knit community. Whatever, the goal, they focused mostly on unity. |
| Leadership | It is a close-knit temple committee. Same leadership since 2005. | Members think they are well consulted. They are happy with leadership. Seems mostly democratic. |
| O & M | They performed collective actions for improved water management. | Collected BDT 4,150 yearly from members. When needed they employ self-labor and raise fund. |
| Networking | Sound knowledge about potential support they can get from DAE, DLS and LGIs. | It seems that leadership is well-shaped to network with GoB organisations and LGIs. |
| Sustainability | They are very confident about sustaining as WMG. They have been working as a coherent group since 2005. | They are more a temple committee who are ready to work jointly to solve any problems; WMG is just a label for them. |
| Functionality | They have the potential to evolve / sustain. | Leadership & members need to strengthen vision. |
Participatory Monitoring[edit | edit source]
Through participatory monitoring of both WMGs (from October 2016 to October 2019) and WMAs (from May to October 2019), WMGs and WMAs had rich experience of using monitoring information to make decisions. Working papers WP8 are a series of reports using information collected through participatory monitoring at WMG level to present self-assessments by WMGs of their performance against potential targets at outcome level – and actions required to move towards full achievement of the targets.
| 28-Feb-17 | WP 8A | Participatory Monitoring October - November 2016 |
| 20-Nov-17 | WP 8B | Participatory Monitoring April-May 2017 |
| 07-Apr-18 | WP 8C | Participatory Monitoring October -November 2017 |
| 24-Jul-18 | WP 8D | Report on Participatory Monitoring April/May 2018 |
| 26-Jan-19 | WP 8E | Report on Participatory Monitoring October 2018 |
| 10-Jul-19 | WP 8F | Report on Participatory Monitoring April 2019 |
| 15-Dec-19 | WP 8G | Report on Participatory Monitoring October 2019 |
Working papers WP10 use information collected through participatory monitoring at WMA level to present self-assessments by WMAs of their performance against potential targets at outcome level.
| 05-Aug-19 | WP 10A | WMA Participatory Monitoring to May 2019 |
| 24-Dec-19 | WP 10B | WMA Participatory Monitoring to October 2019 |
Post-Project Monitoring[edit | edit source]
Whilst the closure of the WMG tracker was because of resource constraints within the TA team, WMAs - the key players for polder water management - had indicated their interest in continuing to operate and use a streamlined version of the self-monitoring system beyond the end of TA involvement. In addition, a meeting held on 20th August 2019 with representatives of EKN/BWDB/TA supported the continuation and suggested that the views of potential stakeholders and options for handover should be examined. During this review, BWDB’s Chief Water Management (CWM) and WMO representatives agreed that the monitoring should continue, but because of resource constraints, CWM requested that WMOs take the major responsibility for data collection and management. The WMOs were willing, but unable to handle large amounts of data. To minimise data processing and reporting, an ICT-based data collection and reporting system was proposed, with WMGs and WMAs collecting and entering data on their status and performance.
Self-assessment of WMG performance[edit | edit source]
WMAs and WMGS are familiar with the process of participatory monitoring. For three years (from October 2016 to October 2019), WMAs and WMGs have assessed their progress and performance against potential development targets - and have used this information in meetings to decide on interventions to lead to a full achievement of those targets. During this participatory monitoring, Blue Gold worked with the WMOs to evolve the process and to observe how the output was used to encourage internal reflection by the WMO – with actions corresponding with the targets. With this experience, there is reasonable confidence that WMOs will continue the self-monitoring using the mobile App. The presence of the TA team in 2020/21 (at a reduced level) will allow some hands-on assistance to establish a quarterly monitoring routine - led and owned by the WMAs.
In January 2020, an IT solution provider, mPower Social Enterprises of Dhaka[Notes 2], was contracted to develop a web-based system for collating and organising data about the status and progress of activities collected and entered by WMGs and WMAs - and for it to be automatically uploaded to a web-platform. The system has been designed so that WMOs can collect data through, and view results in Bangla on, their mobile phones. The mobile application is used for data collection each quarter. Through self-monitoring, the WMAs and WMGs are able to examine their performance in three key areas of their organizations:
- Legality – Does the WMA, and its constituent WMGs, still comply to the formal requirements of its registration: are there timely elections, are accounts ready for audit, are annual general meetings (AGMs) held?
- Legitimacy – is the WMA recognised within the polder; does it represent 70% of active WMGs and well-connected to LGI, BWDB and others?
- Functionality – Does the WMA do what it is set-up for: ensure catchment management plans are current, and that water management activities are carried out?
The performance assessments are colour coded, and WMAs can also access status reports of their constituent WMGs as well as the status reports of other WMAs (for purposes of comparison).
At the time of writing (in March 2021), two rounds of WMO self-assessment have by been conducted using the IT-based mobile application, and a third is planned for Q1 2021. All WMOs were trained in the use of the App, and most WMOs - 35 out of 36 WMAs and 507 out of 511 WMGs - participated in the self-monitoring exercise. The feedback from the WMOs has been mostly positive, they find the system simple and intuitive. The two main challenges were that: (a) the App couldn’t be installed on some of the WMO mobiles because the configuration was incompatible; and (b) the signal from the mobile network was not always sufficiently strong to operate the App.
The polder teams have reviewed the monitoring results with a number of WMAs and WMGs during meetings for other purposes, during which they encourage the WMOs to reflect on the results and plan follow-up actions. From March 2020 and through the first half of 2021 (when permitted by government restrictions to minimise the spread of COVID), WMA capacity building workshops used the self-monitoring outcomes to help WMAs analyse their strengths and the areas for improvement.
Reports (in English) on the findings of the first two self-monitoring exercises are available in the File Library:
| 28-Dec-20 | WP 11A | WMO Self-Monitoring Q3- 2020 |
| 25-Mar-21 | WP 11B | WMO Self-Monitoring Q4- 2020 |
| WP 11C | WMO Self-Monitoring Q1- 2021 |
Notes[edit | edit source]
- ↑ The Master Files on these 12 polders were incorporated into the Polder Development Plans (PDPs). For the remaining 10 Blue Gold polders the information assembled for the Master Files was published only in the PDPs.
- ↑ mPower Social Enterprises had worked earlier with Blue Gold in developing the overall MIS system so were already familiar with the project and its aims and objectives.
References[edit | edit source]
See more[edit | edit source]
| Blue Gold Wiki | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|