View source for 34 Monitoring and evaluation

=== 4. Monitoring and evaluating outcomes/results ===
A main purpose of M&E was to measure and assess performance to manage the development results - outputs and outcomes. M&E at this level, therefore, helped to improve performance and achieve results. It was one thing to implement activities and report about the many quantitative achievements, and another for these achievements to cause the necessary changes or livelihood improvements within the coastal communities. Outcomes of BGP were measured through the organisational development of WMOs as well as increases in agricultural production and profitability.

==== Participatory outcomes monitoring for WMOs ====
Refer to Chapter 35 “[[35 Management Information System#Self-assessment of WMG performance|Self-assessment of WMG performance]]”

==== Outcome surveys for monitoring and evaluation ====
Here we examine outcomes resulting using three different instruments: WMG outcome surveys, DAE agricultural data and earth observation data.

===== WMG outcome surveys 2018, 2019, and 2021 =====
To evaluate the outcomes of BGP interventions through the changes in production and profitability resulting from changes in cropping patterns and improved farming at WMG level, BGP conducted series of WMG surveys in 2018 and 2019. In 2021, BGP team implemented another WMG survey, that made available data on the changes in production and profitability of main crops over the three seasons 2017/18, 2018/19 and 2020/21. An independent consultancy firm was responsible for data collection, analysis and reporting of these surveys.

Before carrying out the first full WMG survey, a pilot study was conducted in P2 in Satkhira and P43/2B in Patuakhali by the TA team and an independent economic analyst appointed by EKN. Based on the findings of the study of the two pilot polders, the study team made some estimates of changes in profitability and income, along with suggestions for further data gathering and analysis to fill gaps in data from these two polders and for assessment of benefits in other polders of BGP. Along with this pilot study, BGP conducted a full phase study to understand the economic changes in all phase I &II polders (P22, P26, P29, P30, P43/1A, P43/2D, P43/2E and P43/2F, P31 Part, P43/2A, P43/2B, P55/2A, P55/2C and P2 & 2Ext.) except polder 2 & 2 Ext. and P43/2B as we had enough information of these two polders from the pilot study. The report on the 2018 WMG Survey was submitted in October 2018.

The WMG survey was repeated in 2019 with 509 WMGs in 22 polders. These surveys compared the pre-project with the present situation of the water management system, along with changes in land use and yields for the main crops, land tenure arrangements, employment and income generation in the three main seasons (''rabi'', ''kharif-1'' and ''kharif-2''). Based on this data Technical Reports TR 25 and TR 26 were prepared and shared with all stakeholders.

A final WMG survey was conducted in 2021. The rationale for planning another round of WMG survey were: WMG survey 2019 was conducted when 53% of the construction works were completed. BGP conducted the impact assessment/endline survey 2020.  This survey was conducted when 72% of the construction works were completed. Data shows that the production of ''aman'' 2019 and ''boro'' 2020 were hampered significantly due to pest/disease attract and uneven weather conditions. So, with a bad production year in the endline survey was not able to cover the full outcomes and impact scenarios of BGP interventions. The 2021 WMG survey was therefore be extended to include all registered WMGs (509) of BGP and 1018 households (2 households per WMG) covering changes in household level in a normal production year compared with the start of BGP for a limited number of key indicators. As a result, along with earlier WMG surveys and end-line survey 2020, WMG survey 2021 will gave a complete outcomes and impact scenario of BGP interventions.

In the WMG surveys, the outcome indicators were carefully identified to enable the assessment of perceptions and responses of the target groups to the program activities and outputs. These surveys helped the implementers to understand the level of acceptance (adaptation) of the expected outputs among the communities of coastal areas. Outcome indicators therefore measured changes in responses of coastal communities particularly concerning the agricultural production and profitability of the specific interventions of BGP. The focuses were on the changes in production, profitability, income and employment generation through adopting new technologies. A thorough analysis of this information at this level showed whether the program was in the right direction or the strategies have some problems. The questionnaires, database of the surveys and all related reports are available in the MIS of BGP.

The reports on these studies include:

* 2018 [[:File:TR25 Outcomes report WMG survey 2018 29oct 18.pdf|TR 25: Improving the Productivity of Land in Coastal Bangladesh: Outcomes of Blue Gold Program Interventions 2013-2018]]
* 2019 [[:File:TR26 outcomes report WMG survey 2019 final 18nov 19 edited.pdf|TR 26: Improving the Productivity of Land in Coastal Bangladesh: Outcomes of Blue Gold Program Interventions 2013-2019]]  
* 2021 [[:File:TR29 outcomes report 2021 11nov 21.pdf|TR 29: Improving the Productivity of Land in Coastal Bangladesh: Outcomes of Blue Gold Program Interventions 2013-2021]].

===== Comparative agricultural performance<ref group="Notes">Using DAE production data collected at ‘Block’ level, the information was reorganized to provide data for the 22 Blue Gold polders over the period 2012-13 to 2020-21</ref> =====
DAE, as one of the two implementing organizations, plays a vital role in increasing agricultural production and profitability through educating farmers on technologies for improved crop production and by utilizing the benefit of improved water management and organizational strength of the WMOs. DAE assembles agricultural data for administrative units, starting with ‘Block level’ (on average there are three Blocks per Union) where DAE Sub-Assistant Agricultural Officers (SAAOs) are responsible for data collection. The data is then aggregated at Upazila and District levels and reported to DAE’s Director Field Services. This means compiling data from all Blocks within a specific polder, on an areal ''pro rata'' basis where the Block is divided between on one or more polders. BGP restructured those preserved data as a secondary source of information and analysed the data to compare practices and land utilization pattern, changes in cropping intensity and yield over the period 2012/13 to 2020/21.
{| class="wikitable"
|+Table 34.4: DAE data analysis of BGP polders
!
! colspan="9" |Analysis in 2021 (TR  28)
! rowspan="3" |Upazila
|-
!
! colspan="5" |Analysis in 2017 (TR  22)
! colspan="4" |
|-
!Polder
!2012/13
!2013/14
!2014/15
!2015/16
!2016/17
!2017/18
!2018/19
!2019/20
!2020/21
|-
!'''Khulna'''
!
!
!
!
!
!
!
!
!
!
|-
!P22
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Paikgacha
|-
!P26
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Dumuria
|-
!P29
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Dumuria, Batiaghata
|-
!P30
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Batiaghata
|-
!P31-part
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Batiaghata
|-
!'''Satkhira'''
|
|
|
|
|
|
|
|
|
|
|-
!P2 and P2 Ext.
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Satkhira sadar &  Assasuni
|-
!'''Patuakhali'''
|
|
|
|
|
|
|
|
|
|
|-
!P43/2A
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Patuakhali Sadar (PS)
|-
!P43/2D
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|PS
|-
!P43/2E
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|PS
|-
!43/2B
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Galachipa, PS &  Amtali
|-
!43/1A
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Amtali
|-
!P43/2F
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Y
|Amtali
|}
In this analysis 12 polders were taken as representative of 22 polders of BGP. The selected polders were polder 2 & 2 ext. from Satkhira, polder 22, 26, 29, 30, 31-part from Khulna, and polder 43/1A, 43/2A, 43/2B, 43/2D, 43/2E, 43/2F from Patuakhali. The study covered 53 blocks under 8 Upazilas (see the table).  These exercises have been done in two stages. In the first stage in 2017, BGP analyzed the data from 2012-13 to 2016-17 and prepared a report (TR 22). In the second stage in 2021, BGP updated the analysis that covered the whole project period 2012-13 to 2020-21 and prepared a report (TR 28). In the second stage an independent consultant was responsible to conduct the survey.  The questionnaire, database of the surveys and all related reports are available in the MIS of BGP.

The reports on these studies include:

* 2017 [[:File:TR22 Agric changes 2013 to 2017 v2 25mar 18.pdf|TR 22: Agriculture Improvements/changes in Blue Gold Polders 2012-13 to 2016-17]]
* 2021 [[:File:TR28 Agricultural Changes 2013 20 v 23sep 21.pdf|TR 28: Agriculture Improvements/changes in Blue Gold Polders 2012-13 to 2020-21]]

==== Remote Sensing/Earth observation survey ====
To evaluate the outcomes of BGP interventions through the changes in the agricultural productivity with cropping intensity and changes in the cultivated area as main indicators, BGP took another survey by making use of Earth Observation (EO) methods (combining Landsat and Sentinel-I). An efficient organisation on EO, Satelligence was contracted for doing this work. The objectives of this survey were 1) to create a baseline for agricultural productivity, 2) show changes in agricultural productivity - which could be related to BGP interventions, and 3) make agricultural productivity information accessible and understandable for the BGP technical team to use for further (intervention) planning. Performance of the polders was estimated for the start of BGP (2011-2015) and two years during the project (2017 and 2018). A report was prepared based on this remote sensing data and shared with all the stakeholders of the BGP.

To update this work BGP contracted another reputed organisation on EO, Terrasphere. They provided analysis and map of agricultural productivity to make a comparison between 2011 and 2020/21 for 22 polders of BGP. A report was prepared based on the analysis of the updated remote sensing data. The database of the surveys and all related reports are available in the MIS of BGP.

The reports on these studies include:

* 2018 [[:File:Satelligence final report 9jul 19.pdf|Earth Observation for Monitoring and Evaluation of Blue Gold Interventions (2011-2017/18)]] (Satelligence)
* 2021 [[:File:10jun 21 TerraSphere satellite mapping 2011 21.pdf|Satellite Mapping of Blue Gold Polders 2011-2021]] (Terrasphere)