Differences

This shows you the differences between two versions of the page.

--- the_capri_data_base [2020/04/25 06:38] – matsz
+++ the_capri_data_base [2022/11/07 10:23] – external edit 127.0.0.1
@@ Line 39: / Line 39: @@
 ==== Technology variants for production activities ====
-For most activities there are two technologies available, typically a low and a high yield variety. Usually they are defined to cover each 50% of the activity level observed in ex post data, but with some particularities in the sugar sector (see //‘\sugar\techf.gms’//).
+For most activities there are two technologies available, typically a low and a high yield variety. Usually they are defined to cover each 50% of the activity level observed in ex post data, but with some particularities in the sugar sector (see //‘/sugar/techf.gms’//).
@@ Line 95: / Line 95: @@
 ** COCO1 module: **\\
-Prepare national database for all EU27 Member States the Western Balkan Countries, Turkey and  Norway.\\ \\ It is basically divided into three main parts:\\
+Prepare national database for all EU27 Member States the Western Balkan Countries, Turkey and  Norway.
+It is basically divided into three main parts:\\
   *  A data import “part” that is not a single “module” but rather a collection activity to prepare a large set of very heterogeneous input files
   * Including and combining these partly overlapping input data according to some hierarchical overlay criteria, and
@@ Line 109: / Line 111: @@
 ** Figure 2: Overview on key elements in the consolidation of European data at the Member state level (in coco1.gms) **
-{{:fig02.png?nolink|}}
+{{:figure_02.png?nolink|}}
 Source: Own illustration
@@ Line 121: / Line 123: @@
 ** Table 3: Data items and their main sources **
 ^ Data items ^ Source ^
-| Activity levels | Eurostat: Crop production statistics, Land use statistics, herd size statistics, slaughtering statistics, statistics on import and export of live animals\\ \\ For Western Balkan Countries and Turkey: Eurostat supplemented with national statistical yearbooks, data from national ministries, FAOstat production statistics and others |
+| Activity levels | Eurostat: Crop production statistics, Land use statistics, herd size statistics, slaughtering statistics, statistics on import and export of live animals For Western Balkan Countries and Turkey: Eurostat supplemented with national statistical yearbooks, data from national ministries, FAOstat production statistics and others |
-| Production, farm and market balance positions | Eurostat: Farm and market balance statistics, crop production statistics, slaughtering statistics, statistics on import and export of live animals\\ \\ For Western Balkan Countries and Turkey: Eurostat supplemented with national statistical yearbooks, data from national ministries,  FAOstat production statistics and others |
+| Production, farm and market balance positions | Eurostat: Farm and market balance statistics, crop production statistics, slaughtering statistics, statistics on import and export of live animals For Western Balkan Countries and Turkey: Eurostat supplemented with national statistical yearbooks, data from national ministries,  FAOstat production statistics and others |
-| Sectoral revenues, costs, and producer prices | Eurostat: Economic Accounts for Agriculture (EAA) and price indices for gap filling, otherwise unit value calculation\\ \\ For Western Balkan Countries and Turkey: Supplemented with national statistical yearbooks, data from national ministries, results from AgriPolicy, FAOstat price statistics |
+| Sectoral revenues, costs, and producer prices | Eurostat: Economic Accounts for Agriculture (EAA) and price indices for gap filling, otherwise unit value calculation For Western Balkan Countries and Turkey: Supplemented with national statistical yearbooks, data from national ministries, results from AgriPolicy, FAOstat price statistics |
 | Consumer prices | Derived from macroeconomic expenditure data (Eurostat, supplemented with UNSTATS) and food price information from various sources |
 | Output coefficients | Derived from production and activity levels, engineering knowledge |
@@ Line 133: / Line 135: @@
 ** Table 4: Temporal coverage of national data by region **
 ^ Member State ^ Range ^
-| EU15 Member States without Germany | 1984 – 2014 |
+| EU15 Member States without Germany | 1984 – 2018/2019 |
-| Germany and (12) New Member States | 1989 – 2014 |
+| Germany and (12) New Member States | 1989 – 2018/2019 |
-| Western Balkan (WB) Countries and Turkey | 1995 – 2014 |
+| Western Balkan (WB) Countries and Turkey | 1995 – 2018/2019 |
-| Norway | 1984 – 2014 |
+| Norway | 1984 – 2017 |
 === Eurostat data ===
@@ Line 162: / Line 164: @@
 ** Second step: data selection and code mapping **
 The second step is data selection and code mapping performed by the GAMS program //‘coco_input.gms’.// Cross sets linking Eurostat codes to COCO codes define the subset of data series subsequently used.
 The mapping rules are collected in two sub-programs called by //‘coco_input.gms’,// for example:
-  * //‘gams\coco\ eurostat_agriculture_mapping.gms’// for the tables from Eurostat’s “Agriculture and Fisheries” Statistics
+  * //‘gams/coco/ eurostat_agriculture_mapping.gms’// for the tables from Eurostat’s “Agriculture and Fisheries” Statistics
   * //‘eurostat_ econfinc_mapping.gms’// for the tables from Eurostat’s “Economy and Finance” Statistics
-Example from file //‘Eurostat _agriculture_mapping.gms’//
+Example from file //‘Eurostat _agriculture_mapping.gms’//. The results of the program run are gdx-files loaded by files (e.g. coco/coco1_eurostat.gms) which are in turn loaded by coco1.gms or coco2.gms.
+<code fortran>
+SET EcoActMAP(ASS_COLS,ASS_ROWS,eco_act_ori_eurostat) "mapping" /
+EAAP.CERE. aact_eaa01_01000_PROD_PP_MIO_EUR
+EAAP.SWHE. aact_eaa01_01110_PROD_PP_MIO_EUR
+EAAP.DWHE. aact_eaa01_01120_PROD_PP_MIO_EUR /;
-| SET AgriProdOriEurostat / \\ apro_acs_a_C1000_AR "CEREALS-EXCLUDING RICE-AREA"\\ apro_acs_a_C1110_AR "COMMON WHEAT AND SPELT - AREA" \\ \\ SET AgriProd_MAP(ASS_COLS,ASS_ROWS,AgriProdOriEurostat) / \\ CERE.LEVL. apro_acs_a_C1000_AR \\ SWHE.LEVL. apro_acs_a_C1110_AR |
-The results of the program run are gdx-files loaded by files (e.g. coco\coco1_eurostat.gms) which are in turn loaded by coco1.gms or coco2.gms.
+SET AgriProdMAP(ASS_COLS,ASS_ROWS,agri_prod_ori_eurostat) "mapping" /
+CERE.LEVL.( apro_cpnh1_C1000_AR,apro_cpnh1_h_C1000_AR)
+SWHE.LEVL.( apro_cpnh1_C1110_AR,apro_cpnh1_h_C1110_AR)
+SWH1.LEVL.( apro_cpnh1_C1111_AR,apro_cpnh1_h_C1111_AR) /;
+</code>
 === Western Balkan Countries and Turkey ===
@@ Line 179: / Line 192: @@
 For those countries Eurostat data need completion in almost every area which is handled in country specific xls files. The structure of these supplementary Excel country sheets and the definitions of the data are tailored to COCO. The resulting sheets in these xls files are uniform across countries, in order to ease data extraction for the modelling part by applying macros. However, each national information system has its own peculiarities and hence, not all data are fully harmonised across countries. Various sources are assessed and combined in a case by case manner: Eurostat data, if already available and plausible, are handled as the preferred data source. Data collected from the national statistical yearbooks have second priority, followed by expert data collected in from earlier projects. Finally FAO data provides often the fall-back solution for any remaining missing time series.
-The final sheet in each of these country specific xls files is the interface to the GAMS programing world of COCO. An Excel macro “SELECT_data_all” collects the time-series compiled in other sheets and puts them into this final sheet with the appropriate COCO code. Another macro finally exports the numbers into text files like “dat\coco\bosnia_coco.gms”. Because the xls file are quite complex due to various linkages, we do not read directly from them. This avoids unplanned changes and permits convenient tracing of data changes via the CAPRI versioning system svn.
+The final sheet in each of these country specific xls files is the interface to the GAMS programing world of COCO. An Excel macro “SELECT_data_all” collects the time-series compiled in other sheets and puts them into this final sheet with the appropriate COCO code. Another macro finally exports the numbers into text files like “dat/coco/bosnia_coco.gms”. Because the xls file are quite complex due to various linkages, we do not read directly from them. This avoids unplanned changes and permits convenient tracing of data changes via the CAPRI versioning system svn.
 === Supplementary data for Romania and Bulgaria ===
@@ Line 190: / Line 203: @@
   * For all regions FAO data (mapped in the context of module “global database” to CAPRI codes and hence consistent across modules) serve as a fall back option under certain conditions, defined in the code. This fall back function of FAO data has gained in importance since Eurostat discontinued the publication of most market balances since 2014. In some cases also activity level (area) information may be taken from FAO.
-  * Some particular data like disaggregate data on herds of chicken, ducks, turkeys and geese are compiled in a separate include file dat\coco\fao_add.gms because these data types are usually not loaded for global database.
+  * Some particular data like disaggregate data on herds of chicken, ducks, turkeys and geese are compiled in a separate include file dat/coco/fao_add.gms because these data types are usually not loaded for global database.
 === Other additional input data ===
-COCO1: Biofuels
+COCO1: Biofuels FIXME (most links are not working anymore, remove or re-link)
   * Production, market balance and feedstock quantities for biodiesel and bioethanol are collected from a multitude of sources:
@@ Line 221: / Line 234: @@
   * Market balances for casein and whey powder were only available on EU level from ZMP, Bonn, which was closed down in 2009.
-  * DG Agri partly completes gaps in Eurostat series and offers this consolidated database for download. This is used to close gaps in gams\coco\coco1_eurostat.
+  * DG Agri partly completes gaps in Eurostat series and offers this consolidated database for download. This is used to close gaps in gams/coco/coco1_eurostat.
 COCO1: Producer prices for cotton
@@ Line 235: / Line 248: @@
 COCO1: Land use data
-The raw data on land use are currently prepared outside the CAPRI system. Source code and input files are available at EuroCARE, Bonn (R:\Coco_input\land_use). Relevant (raw) information is stored in dat\coco\landuse_data_and_sets.gdx. The data base comprises information on land use classes from various sources, which are again partly discontinued but useful for the early years:
+The raw data on land use are currently prepared outside the CAPRI system. Source code and input files are available at EuroCARE, Bonn (R:/Coco_input/land_use). Relevant (raw) information is stored in dat/coco/landuse_data_and_sets.gdx. The data base comprises information on land use classes from various sources, which are again partly discontinued but useful for the early years:
   * REGIO - Eurostat, land use, REGIO domain( NUTS2 level - yearly, 1984-2014)
@@ Line 243: / Line 256: @@
   * FAO -  area.xls(MS level - yearly, 1984-2016)
   * MCPFE (Ministerial Conference on the Protection of Forests in Europe), jointly published by FAO and UNECE (MS level - 1990, 2000, 2005, 2010, 2015)
-  * FSS - Eurostat, FSS(NUTS2 level - 1990, 1993, ..., 2007, 2010, 2013), only added in coco1\landuse
+  * FSS - Eurostat, FSS(NUTS2 level - 1990, 1993, ..., 2007, 2010, 2013), only added in coco1/landuse
   * UNFCCC (1990-2016), also covers land transitions and settlement data. Official data for LULUCF accounting, merged with other data in coco1_landuse.
@@ Line 344: / Line 357: @@
 The file handling the previous actions is ‘coco1_finish_agriprod.gms’:
-{{:SS05.png?nolink|}}
+{{:ss05_neu.jpg?nolink|}}
 The previous code snippet also shows for the interested reader two frequently used debugging devices:
-  - The key parameters at a certain point in the program flow (above: p_agriProd, p_agriPri, p_ecoAct) are copied to a debugging parameter “debug” (better name would be: “p_debug”). At the end of a coco1 run (or if desired also at this point) the parameter is unloaded into a file “results\coco\debug\debug_%MS%.gdx” such that the various assignments, corrections, deletions that have occurred up to a certain program line may be inspected in one file.
+  - The key parameters at a certain point in the program flow (above: p_agriProd, p_agriPri, p_ecoAct) are copied to a debugging parameter “debug” (better name would be: “p_debug”). At the end of a coco1 run (or if desired also at this point) the parameter is unloaded into a file “results/coco/debug/debug_%MS%.gdx” such that the various assignments, corrections, deletions that have occurred up to a certain program line may be inspected in one file.
-  - The next command “$batinclude “util\debug” %system.fn% %system.incline%  unloads the whole memory, incuding all parameters but also sets and other symbols, at this point into a debugging file in the gams\temp folder. This may be useful to analyse “difficult” cases of debugging.
+  - The next command “$batinclude “util/debug” %system.fn% %system.incline%  unloads the whole memory, incuding all parameters but also sets and other symbols, at this point into a debugging file in the gams/temp folder. This may be useful to analyse “difficult” cases of debugging.
-  -
 Finally the biofuel sector is prepared.
@@ Line 359: / Line 372: @@
 //Biofuel production//
-There is no differentiation made between fuel- or non-fuel (undenatured or denatured) quantities in production, import and export positions of ethanol. But the consumption position of ethanol is differentiated in fuel-ethanol consumption and non-fuel-ethanol consumption. Hence data on fuel and non-fuel production and consumption of ethanol was required. In the case of biodiesel this differentiation is irrelevant. The ex-post data on biofuel production are coming from diverse sources which is unavoidable to complete the data for years as of 2002 up to the present, if necessary with the help of second and third best solutions or assumptions (compare //biofuel\prepare_biofuel_data.gms//).
+There is no differentiation made between fuel- or non-fuel (undenatured or denatured) quantities in production, import and export positions of ethanol. But the consumption position of ethanol is differentiated in fuel-ethanol consumption and non-fuel-ethanol consumption. Hence data on fuel and non-fuel production and consumption of ethanol was required. In the case of biodiesel this differentiation is irrelevant. The ex-post data on biofuel production are coming from diverse sources which is unavoidable to complete the data for years as of 2002 up to the present, if necessary with the help of second and third best solutions or assumptions (compare //biofuel/prepare_biofuel_data.gms//).
 The overlay considers data availability and consistency across sources:
@@ Line 370: / Line 383: @@
 //Feedstock demand//
-In addition to market balances for the fuels the CAPRI data base requires the shares of the raw products on the production of biodiesel and bioethanol at the level of CAPRI products. For bioethanol, this information is partly provided by the DG Agri balances, hence this has been selected to be the major source. The detailed recording follows from the existence of support measures for distillation of wine, fruits and potatoes which triggered a detailed monitoring of ethanol markets. However, for biodiesel the statistical sources are scarce. It turns out that the most consistent estimates for EU regions are apparently produced by USDA services, covering rape, sunflower, soya, palm oil but also used cooking oils, tallow and other oils. As these data do not cover single MS an estimation procedure has been devised (in //biofuel\calc_feedstock_shares.gms//). The initialisation of this estimated feedstock composition relied on the observed increase in INDM according to Eurostat (or more precisely the COCO initialisation when entering //‘prepare_biofuel_data.gms’//) which is assumed to be the main source to “cut out” the required biofuel processing quantities (BIOF) by MS from market balances that so far did not include BIOF.
+In addition to market balances for the fuels the CAPRI data base requires the shares of the raw products on the production of biodiesel and bioethanol at the level of CAPRI products. For bioethanol, this information is partly provided by the DG Agri balances, hence this has been selected to be the major source. The detailed recording follows from the existence of support measures for distillation of wine, fruits and potatoes which triggered a detailed monitoring of ethanol markets. However, for biodiesel the statistical sources are scarce. It turns out that the most consistent estimates for EU regions are apparently produced by USDA services, covering rape, sunflower, soya, palm oil but also used cooking oils, tallow and other oils. As these data do not cover single MS an estimation procedure has been devised (in //biofuel/calc_feedstock_shares.gms//). The initialisation of this estimated feedstock composition relied on the observed increase in INDM according to Eurostat (or more precisely the COCO initialisation when entering //‘prepare_biofuel_data.gms’//) which is assumed to be the main source to “cut out” the required biofuel processing quantities (BIOF) by MS from market balances that so far did not include BIOF.
 A special case was palm oil, as the CAPRI database (COCO) doesn’t cover an industrial use position for this product so far. EUROSTAT-COMEXT delivers data on import and export quantities of crude palm oil (HS 151110) for EU Member states. Thereby an increase of palm oil imports was observed within the relevant ex post period (2002-2005). Thus the following assumptions were made to derive approximated values for palm oil processing to biodiesel: (a) Import quantities minus export quantities are equal to domestic consumption of palm oil as domestic production in European Member states can be neglected. (b) The average aggregated consumption quantity of palm oil before 2002 was assumed to be completely used for human consumption as no significant biodiesel consumption took place. By subtracting this constant share of human consumption from the observed consumption quantities after 2002 gave an estimate for the quantities used for industrial processing
@@ Line 406: / Line 419: @@
 //Fuel prices and taxes//
-For a specification of processing-, biofuel supply- and demand-functions in the base year, ex post prices are required. Furthermore, given the structure of the CAPRI market module (described in Section [[Market module for agricultural outputs]] ), a differentiation of producer, consumer and import price is also needed. These differentiated prices are not covered in any statistical database for biofuels but they can be derived indirectly by given information on taxes, tariffs and subsidies from the world market price which is available. Thus beside ex post prices information on consumer (excise) taxes, import tariffs and further subsidies are required. The AgLink-Cosimo database includes ex post world market prices for ethanol and biodiesel. This price was taken as the base value to calculate the differentiated prices in the respective countries. The import tariffs for ethanol and biodiesel were also taken from the AgLink-Cosimo database. As the consumer taxes for ethanol and biodiesel in most instances correspond to a reduced excise tax on fossil fuels the consumer taxes for gasoline and diesel were taken as a base value. This tax information was acquired from EurActiv((FIXME [[http://www.euractiv.com/en/taxation/fuel-taxation/article-117495]], 20.07.2009.)) where levels of diesel and petrol taxation in 2002 are published for European Member states. For the required time period (2002-2005) taxation levels were calculated with respect to COM(2002)410((Proposal for a Council Directive amending Directive 92/81/EEC and Directive 92/82/EEC to introduce special tax arrangements for diesel fuel used for commercial purposes and to align the excise duties on petrol and diesel fuel (COM(2002)410).)) which set minimum excise tax rates for non-commercial diesel and petrol since 2006. To identify the excise tax exemptions and producer subsidies, if existent, for the single Member states the obligatory ‘Member States reports on the implementation of Directive 2003/30/EC of 8 May 2003 on the promotion of the use of biofuels or other renewable fuels for transport’ were consulted which are published by the Commission(([[http://ec.europa.eu/energy/renewables/biofuels/ms_reports_dir_2003_30_en.htm]].)). Three different types of tax regulations for biofuels were identified which are applied among the different Member states: an absolute tax for biofuels, an absolute reduction of the excise tax on fossil fuels and a relative reduction of the excise tax on fossil fuels. All differentiated in taxation for blended biofuels or pure biofuels. Based on this information the different ex post prices for the period 2002-2005 were recalculated. As the envisaged biofuel demand function will be a function of (among other variables) the relation between fossil fuel consumer prices and biofuel consumer prices the acquisition of fossil fuel prices was required additionally. To hold consistency between the biofuel and fossil fuel prices the price information for fossil fuels were also taken from the AgLink-Cosimo database which provides EU market prices for diesel and petrol. For the recalculation of consumer prices in individual Member states the already collected taxation levels for fossil fuels were applied. Because there exists a significant difference between the physical energy content and the density of biodiesel, ethanol, petrol and diesel a direct comparison of prices (in €/t) is not possible. For this reason the prices as well as the taxation levels were converted into Euro per ton oil equivalent (toe).
+For a specification of processing-, biofuel supply- and demand-functions in the base year, ex post prices are required. Furthermore, given the structure of the CAPRI market module (described in Section [[scenario simulation#Market module for agricultural outputs]] ), a differentiation of producer, consumer and import price is also needed. These differentiated prices are not covered in any statistical database for biofuels but they can be derived indirectly by given information on taxes, tariffs and subsidies from the world market price which is available. Thus beside ex post prices information on consumer (excise) taxes, import tariffs and further subsidies are required. The AgLink-Cosimo database includes ex post world market prices for ethanol and biodiesel. This price was taken as the base value to calculate the differentiated prices in the respective countries. The import tariffs for ethanol and biodiesel were also taken from the AgLink-Cosimo database. As the consumer taxes for ethanol and biodiesel in most instances correspond to a reduced excise tax on fossil fuels the consumer taxes for gasoline and diesel were taken as a base value. This tax information was acquired from EurActiv((FIXME [[http://www.euractiv.com/en/taxation/fuel-taxation/article-117495]], 20.07.2009.)) where levels of diesel and petrol taxation in 2002 are published for European Member states. For the required time period (2002-2005) taxation levels were calculated with respect to COM(2002)410((Proposal for a Council Directive amending Directive 92/81/EEC and Directive 92/82/EEC to introduce special tax arrangements for diesel fuel used for commercial purposes and to align the excise duties on petrol and diesel fuel (COM(2002)410).)) which set minimum excise tax rates for non-commercial diesel and petrol since 2006. To identify the excise tax exemptions and producer subsidies, if existent, for the single Member states the obligatory ‘Member States reports on the implementation of Directive 2003/30/EC of 8 May 2003 on the promotion of the use of biofuels or other renewable fuels for transport’ were consulted which are published by the Commission(([[http://ec.europa.eu/energy/renewables/biofuels/ms_reports_dir_2003_30_en.htm]].)). Three different types of tax regulations for biofuels were identified which are applied among the different Member states: an absolute tax for biofuels, an absolute reduction of the excise tax on fossil fuels and a relative reduction of the excise tax on fossil fuels. All differentiated in taxation for blended biofuels or pure biofuels. Based on this information the different ex post prices for the period 2002-2005 were recalculated. As the envisaged biofuel demand function will be a function of (among other variables) the relation between fossil fuel consumer prices and biofuel consumer prices the acquisition of fossil fuel prices was required additionally. To hold consistency between the biofuel and fossil fuel prices the price information for fossil fuels were also taken from the AgLink-Cosimo database which provides EU market prices for diesel and petrol. For the recalculation of consumer prices in individual Member states the already collected taxation levels for fossil fuels were applied. Because there exists a significant difference between the physical energy content and the density of biodiesel, ethanol, petrol and diesel a direct comparison of prices (in €/t) is not possible. For this reason the prices as well as the taxation levels were converted into Euro per ton oil equivalent (toe).
 ===Assigning data to database array===
@@ Line 469: / Line 482: @@
 Two particularities in the pig sector are worth mentioning. The first is that as of 2011 the COCO database includes the herd size of piglets < 20kg (on code PIGL00.HERD) even though there is no explicit activity level “raising of piglets”. Instead the piglets raised are one of the outputs of activity sows with total production of piglets given on code GROF.YPIG. Accordingly we cannot store the process length for raising of piglets in a column for “raising of piglets” but introduce a new code “PIGF.YDAYS” such that in the completed data base we find the relationship PIGF.YDAYS = GROF.YPIG / PIGL00.HERD * 365. Including the piglets turned out useful because it permits to make use of statistical data on the total pigs population which is sometimes available even though pig slaughterings in heads are missing.
-The second pig sector particularity relates to the requirement functions for pigs, stored in the form of a table (//\dat\feed\porkreq.gms//) that relates daily growth to final slaughter weights. For consistency reasons the same table is used to define bounds for the permissible process length.
+The second pig sector particularity relates to the requirement functions for pigs, stored in the form of a table (// /dat/feed/porkreq.gms//) that relates daily growth to final slaughter weights. For consistency reasons the same table is used to define bounds for the permissible process length.
 In the poultry sector we have herd size data for chicken broilers, turkeys, ducks, and geese (yearly average, mainly from FAO) and hens from Eurostat (average of this and last year’s December counting). The first four give the total herd size of poultry for fattening whereas the herd size of hens also equals the activity level. The output coefficient for eggs relies on usable production from the balance sheets divided by the herd size of hens. A replacement rate of 80% is assumed for laying hens. The activity level of poultry fattening is the difference of total produced poultry heads minus slaughtered hens. The output coefficients and production in terms of meat are straightforward to calculate from here. With activity level and aggregate herd size of poultry for fattening being defined it is possible to calculate the implied process length. The information on the shares of chicken broilers, turkeys, ducks, and geese is used to specify technical bounds for the daily growth and process length. In addition the technical literature also permitted to specify typical empty days for cleaning of stables (or seasonality in the case of geese and ducks). The differentiation of poultry for fattening is only maintained temporarily in COCO1 because it helped to use statistical information for the specification of some technical coefficients that strongly depend on the shares of turkeys. Subsequent CAPRI modules (like CAPREG) will only use the COCO results for the aggregate poultry fattening activity (POUF).
@@ Line 528: / Line 541: @@
     * To acknowledge that the Corine Classes may be mapped to several LUCAS categories we multiplied them with the “profiles”, giving the distribution of each Corine category according to the LUCAS classes. In this case, only 26.7% of the “complexCultiv” area is mapped to annual crops, but 7.3% are mapped to “temporary pastures”, 6.4% to “permanent  grassland with sparse tree/shrub vegetation” and so forth. The transformed Corine data often give the most detailed area coverage and thus assume a role as a kind of fall back information in case that other information is missing.
   * **LEVRegio** - Eurostat regional land use data (Eurostat Table: “agr_r_landuse”, discontinued). Inspite of using the same codes as for the national data, the national totals, aggregated from the NUTS2 regions are not always in line with LEVAgriProd. Furthermore a few categories are missing (no inland waters, no other wooded land). However there are few alternative annual series available to regionalise the national data in CAPREG.
-  * **LEVFAO** - Land use data from the resource FAOSTAT domain FIXME ((See [[http://faostat3.fao.org/home/index.html#DOWNLOAD]].)) with annual time series on agricultural land use but also some non agricultural area categories (forest, inland waters, other land, total area).
+  * **LEVFAO** - Land use data from the resource FAOSTAT domain FIXME ((See [[http://faostat3.fao.org/home/index.htmlDOWNLOAD]].)) with annual time series on agricultural land use but also some non agricultural area categories (forest, inland waters, other land, total area).
   * **LEVLucas** – directly using the LUCAS data is an option that has been considered but not implemented in CAPRI so this code is not used at the moment.
   * **LEVLandCov** - Eurostat land cover data for 2009, 2012, 2015 at the MS level. Agricultural land is only distinguished into cropland CROP and grassland GRAS, but 5 nonagricultural areas are neatly aggregating up to the total country (Artificial ARTIF, shrubland (considered similar to “other wooded land” OWL), bare land & wetlands (mapped to “other sparcely vegetated or bare OSPA) and waters WATER.
@@ Line 616: / Line 629: @@
 It should be mentioned that the above representation of the COCO objective function is a quite simplified one: It is evident that the above lacks safeguards against division by zero or very small values which are included in the GAMS code. Furthermore there are different types of gaps which are not reflected above to avoid clutter (Are there gaps in a series with some data or is the series empty? Is the mean based on data or estimated from  \(y_{i,t}^{lo},y_{i,t}^{up}\) ?)
-Equation 4 indicates that accountancy restrictions are added. These restrictions can be balances (land, milk contents, young animals), aggregation conditions, definitions for processing coefficients and yields etc. They are quite similar to those applied for the ex ante trend projections as discussed in detail in Section [[The Regionalised Data Base (CAPREG)]] but the COCO1 accounting identities tend to acknowledge more details or have to establish the data base that is subsequently given for the ex ante trend projections, for example related to the split of high and low yield animal activites (DCOL, DCOH, BULL, BULH, HEIL, HEIH):
+Equation 4 indicates that accountancy restrictions are added. These restrictions can be balances (land, milk contents, young animals), aggregation conditions, definitions for processing coefficients and yields etc. They are quite similar to those applied for the ex ante trend projections as discussed in detail in Section [[The capri data base#The Regionalised Data Base (CAPREG)]] but the COCO1 accounting identities tend to acknowledge more details or have to establish the data base that is subsequently given for the ex ante trend projections, for example related to the split of high and low yield animal activites (DCOL, DCOH, BULL, BULH, HEIL, HEIH):
 {{:code_p.43.png?600|}}
@@ Line 741: / Line 754: @@
 **Figure 3: Overview on main estimations in for the consolidation of national data in Europe (in coco1.gms)**
-{{::figure3.png?600|}}
+{{::figure_03.png?600|}}
 Results are not always fully satisfactory (perhaps impossible given some raw data). For example the resulting prices (unit values) are far from a priori expectations for a number of series, in particular less important ones. This is because, apart from some additional security checks, unit values are by and large considered a free balancing variable calculated to preserve the identity between largely fixed EAA values and fixed production (in coco1_estimb). The priority for EAA values has been reduced somewhat in recent years but a more thorough revision would require to estimate production, market balances and EAA simultaneously rather than consecutively (first $(a)$, then $(c)$ for crops). As this is infeasible for all crops at the same time the whole estimation would need to be split up differently in the crop sector, perhaps first for the aggregates and then within those.
@@ Line 814: / Line 827: @@
 & \cdot PEX_{m,FOPOS,k} \cdot log(PEX_{m,FOPOS,k}/PQ_k)\\
 &-\sum_{m,j,k} PFAC_{m,k} \cdot LOG(PFAC_{m,,k}/PQ_k)\cdot 1000\\
 \end{split}
 \end{align}
@@ Line 821: / Line 833: @@
 Parameters are
-| \(HCOM_{m,j,t}\) |Human consumption, result from COCO1|
+|\(HCOM_{m,j,t}\) |Human consumption, result from COCO1|
-| \(UVAD_{m,j,t\_1}\)	|Consumer price from last simulation of year t+1|
+|\(UVAD_{m,j,t\_1}\)	|Consumer price from last simulation of year t+1|
 |\(CPS_{m,j,k}\)	|Support points for consumer prices |
 |\(DCPS_{m,j,k}\)	|Support points for consumer price changes|
@@ Line 915: / Line 927: @@
   * In some cases it is convenient to have the completed COCO1 results of all countries at hand for comparison purposes and in order to achieve a balanced picture across MS. This is the main motive for the assignments of consumer loss rates (Section 3.2.7.1).
-  * Whenever averages of consolidated data (from COCO1) across several or all MS are involved, a solution in a loop requires certain sequence (such as first solving for non-candidate countries to form the averages that are input to candidate countries) or is better solved in a new module like COCO2. This applies to the expenditure allocation problem (Section [[The Complete and Consistent Data Base (COCO) for the national scale#COCO2: Data Preparation]]), to completions for certain feedstuffs (Section 3.2.7.2, EU averages used due to the scarcity of data), and to corrections of LULUCF coefficients (Section 3.2.7.3). FIXME
+  * Whenever averages of consolidated data (from COCO1) across several or all MS are involved, a solution in a loop requires certain sequence (such as first solving for non-candidate countries to form the averages that are input to candidate countries) or is better solved in a new module like COCO2. This applies to the expenditure allocation problem (Section [[the capri data base#COCO2: Data Preparation]]), to completions for certain feedstuffs (Section 3.2.7.2, EU averages used due to the scarcity of data), and to corrections of LULUCF coefficients (Section 3.2.7.3). FIXME
 ===Assignment of consumer loss rates and nutrient intake per head ===
-Since a number of years diet shift scenarios have increase in importance and therefore the plausibility of per capita consumption projectios and hence their starting values, per capita consumption in the data base. A common yardstick to assess plausibility is nutrient (e.g. calorie) consumption per head where the nutrition literature offers guidance in terms of recommendable as well as “observed” consumption. For nutrition issues it is intake, so consumption after losses, which matters, such that the assignment of these loss rates becomes a critical element of the database. The starting values are due to an FAO study and stored in the \dat folder
+Since a number of years diet shift scenarios have increase in importance and therefore the plausibility of per capita consumption projectios and hence their starting values, per capita consumption in the data base. A common yardstick to assess plausibility is nutrient (e.g. calorie) consumption per head where the nutrition literature offers guidance in terms of recommendable as well as “observed” consumption. For nutrition issues it is intake, so consumption after losses, which matters, such that the assignment of these loss rates becomes a critical element of the database. The starting values are due to an FAO study and stored in the /dat folder
 {{::code_p53.png?600|}}
@@ Line 980: / Line 992: @@
 \begin{align}
 \begin{split}
 &fatngday_r^{HEIF}\\
 &= min \left[ DAYS_{up}^{HEIF},max \left\{ DAYS_{lo}^{HEIF},\frac{BEEF_r^{HEIF}/carcassSh_{HEIF}-startWgt_{HEIF}}{dailyIncrease_r^{HEIF}} \right\} \right]
 \end{split}
 \end{align}
@@ Line 1019: / Line 1029: @@
 After this last include file the completions in module COCO2 are finished and the main output file (coco2_output.gdx) is unloaded. This file is loaded in subsequent modules (main use in CAPREG, but also in CAPTRD for nowcasting and in CAPMOD for update of LULUCF coefficients).
+====Annex: Code lists for the COCO database====
+This section includes detailed code lists, which are in use in the COCO database.
+**Table: Codes used for storing the original REGIO tables in the database and their description (rows)**
+^ Codes used in CAPRI REGIO tables ^ Original REGIO description ^
+| TOTL | Territorial area |
+| FORE | Forest land |
+| AGRI | Utilized agricultural area |
+| GARD | Private gardens |
+| GRAS | Permanent grassland |
+| PERM | Permanent crops |
+| VINE | Vineyards |
+| OLIV | Olive plantations |
+| ARAB | Arable land |
+| GREF | Green fodder on arable land |
+| CERE | Cereals (including rice) |
+| WHEA | Soft and durum wheat and spelt |
+| BARL | Barley |
+| MAIZ | Grain maize |
+| RICE | Rice |
+| POTA | Potatoes |
+| SUGA | Sugar beet |
+| OILS | Oilseeds (total) |
+| RAPE | Rape |
+| SUNF | Sunflower |
+| TOBA | Tobacco |
+| MAIF | Fodder maize |
+| CATT | Cattle (total) |
+| COWT | Cows (total) |
+| DCOW | Dairy cows |
+| CALV | Other cows |
+| CAT1 | Total cattle under one year |
+| CALF | Slaughter calves |
+| CABM | Male breeding calves (\&lt;1 year) |
+| CABF | Female breeding calves (\&lt;1 year) |
+| BUL2 | Male cattle (1-2 years) |
+| H2SL | Slaughter heifers (1-2 years) |
+| H2BR | Female cattle (1-2 years) |
+| BUL3 | Male cattle (2 years and above) |
+| H3SL | Slaughter heifers (2 years and above) |
+| H3BR | Breeding heifers |
+| BUFF | Total buffaloes |
+| PIGS | Total pigs (total) |
+| PIG1 | Piglets under 20 kg |
+| PIG2 | Piglets under 50 kg and over 20 kg |
+| PIG3 | Fattening pigs over 50 kg |
+| BOAR | Breeding boars |
+| SOW2 | Total breeding sows |
+| SOW1 | Sows having farrowed |
+| GILT | Gilts having farrowed for the first time |
+| SOWM | Maiden sows |
+| GILM | Maiden gilts |
+| SHEP | Sheep total) |
+| GOAT | Goats (total) |
+| EUQI | Equidae (total) |
+| POUL | Poultry (total) |
+| OUTP | Final production |
+| CROP | Total crops production |
+| DWHE | Durum wheat |
+| PULS | Pulses |
+| ROOT | Roots and tubers |
+| INDU | Industrial crops |
+| TEXT | Textile fibre plants |
+| HOPS | Hops |
+| VEGE | Fresh vegetables |
+| TOMA | Tomatoes |
+| CAUL | Cauliflowers |
+| FRUI | Fresh fruit |
+| APPL | Apples |
+| PEAR | Pears |
+| PEAC | Peaches |
+| CITR | Citrus fruit (total) |
+| ORAN | Oranges |
+| LEMN | Lemons |
+| MAND | Mandarins |
+| GRAP | Table grapes |
+| WINE | Wine |
+| TABO | Table olives |
+| OLIO | Olive oil |
+| NURS | Nursery plants |
+| FLOW | Flowers and ornamental plants |
+| OCRO | Other crops |
+| ANIT | Total animal production |
+| ANIM | Animal |
+| SHGO | Sheep and goats |
+| ANIP | Animal products |
+| MILK | Milk |
+| EGGS | Eggs |
+| INPU | Intermediate consumption (total) |
+| FEED | Animal feeding stuffs |
+| FDGR | Animal compounds for grazing livestock |
+| FDPI | Animal compounds for pigs |
+| FDPO | Animal compounds for poultry |
+| FODD | Straight feeding stuffs |
+| FERT | Fertilizers and enrichments |
+| ENER | Energy and lubricants |
+| INPO | Other inputs |
+| GVAM | Gross value added at market prices |
+| SUBS | Subsidies |
+| TAXS | Taxes linked to production (including VAT balance) |
+| GVAF | Gross value added at factor costs |
+| DEPM | Depreciation |
+| LABO | Compensation and social security contributions of employees |
+| RENT | Rent and other payments |
+| INTE | Interests |
+| GFCF | Total of gross fixed capital formation |
+| BUIL | Buildings and other structures |
+| MACH | Transport equipment and machinery |
+| GFCO | Other gross fixed capital formation |
+**Table: Codes used for storing the original REGIO tables in the data base and their description (columns)**
+^ Codes used in CAPRI REGIO tables ^ Original REGIO description ^
+| LEVL | Herd size / Area / # of persons |
+| LSUN | Live stock units |
+| PROP | Physical production |
+| YILD | Yield |
+| VALE | EAA position in ECU |
+| VALN | EAA position in NC |
+**Table: Connection between CAPRI and REGIO crop areas, crop production and herd sizes**
+^ SPEL-code ^ REGIO-code ^ REGIO-code ^ REGIO-code ^ REGIO-code ^ Description of SPEL activity ^
+| SWHE | WHEA | CERE | ARAB | | Soft wheat |
+| DWHE | WHEA | CERE | ARAB | | Durum wheat |
+| RYE | | CERE | ARAB | | Rye |
+| BARL | BARL | CERE | ARAB | | Barley |
+| OATS | | CERE | ARAB | | Oats |
+| MAIZ | MAIZ | CERE | ARAB | | Maize |
+| OCER | | CERE | ARAB | | Other cereals (excl. rice) |
+| PARI | RICE | CERE | ARAB | | Paddy rice |
+| PULS | | | ARAB | | Pulses |
+| POTA | POTA | | ARAB | | Potatoes |
+| SUGB | SUGA | | ARAB | | Sugar beet |
+| RAPE | RAPE | OILS | ARAB | | Rape and turnip rape |
+| SUNF | SUNF | OILS | ARAB | | Sunflower seed |
+| SOYA | | OILS | ARAB | | Soya beans |
+| OLIV | | OLIV | PERM | | Olives for oil |
+| OOIL | | OILS | ARAB | | Other oil seeds and oleaginous fruits |
+| FLAX | | | ARAB | | Flax and hemp \*\*\* (faser) \*\*\* |
+| TOBA | TOBA | | ARAB | | Tobacco, unmanufactured, incl. dried |
+| OIND | | | ARAB | | Other industrial crops |
+| CAUL | | | ARAB | | Cauliflowers |
+| TOMA | | | ARAB | | Tomatoes |
+| OVEG | | | ARAB | | Other vegetables |
+| APPL | | | PERM | | Apples, pears and peaches |
+| OFRU | | | PERM | | Other fresh fruits |
+| CITR | | | PERM | | Citrus fruits |
+| TAGR | | VINE | PERM | | Table grapes |
+| TABO | | OLIV | PERM | | Table olives |
+| TWIN | | VINE | PERM | | Table wine |
+| OWIN | | VINE | PERM | | Other wine |
+| NURS | | | PERM | | Nursery plants |
+| FLOW | | | ARAB | | Flowers,ornamental plants, etc. |
+| OCRO | | | ARAB | | Other final crop products |
+| MILK | DCOW | | | | Dairy cows |
+| BEEF | BUL2 | BUL3 | | | Bulls fattening |
+| CALF | CALF | | | | Calves fattening (old VEAL) |
+| PORK | PIG3 | PIG2 | PIG1 | | Pig fattening |
+| MUTM | GOAT | SHEP | | | Ewes and goats |
+| MUTT | GOAT | SHEP | | | Sheep and goat fattening |
+| EGGS | POUL | | | | Laying hens |
+| POUL | POUL | | | | Poultry fattening |
+| OANI | | | | | Other animals |
+| OROO | | | ARAB | | Other root crops |
+| GRAS | GRAS | | | | Green fodder |
+| SILA | GREF | | ARAB | | Silage |
+| CALV | CALV | | | | Suckler cows |
+| RCAL | CABM | CABF | | | Calves, raising |
+| HEIF | H2SL | H2BR | H3SL | H3BR | Heifers |
+| PIGL | SOW2 | | | | Pig breeding |
+| FALL | | | FALL | | Fallow land |
+**Tables: Codes of the input allocation estimation**
+^FADN inputs (FI) ^Label  ^
+| TOIN | total inputs |
+| COSA | animal specific inputs |
+| FEDG | self grown feedings |
+| ANIO | other animal inputs |
+| FEDP | purchased feedings |
+| COSC | crop specific inputs |
+| SEED | seeds |
+| PLAP | plant protection |
+| FERT | fertilisers |
+| TOIX | other inputs (overheads) |
+^CAPRI inputs (CI) used in the reconciliation ^label ^
+| TOIN | total inputs |
+| FEED | feedings |
+| IPHA | other animal inputs |
+| COSC | crop specific inputs |
+| SEED | seeds |
+| PLAP | plant protection |
+| FERT | fertilisers |
+| REPA | repairs |
+| ENER | energy |
+| SERI | agricultural services input |
+| INPO | other inputs |
+. The set of //Other// activities that had been omitted from the econometric estimation:
+  * OTHER={OCER, OFRU, OVEG, OCRO, OWIN, OIND, OOIL, OFAR, OANI}
+. The set of activity groups, and their elements, used in the replacement or missing/negative coefficients
+  * GROUPS = {YOUNG, VEGE, SETT, PULS, PIG, OILS, MILK, MEAT, INDS, HORSE, GOAT, FRU, FOD, FLOWER, DENNY, COW, CHICK1, CHICK2, CHICK3, CERE, ARAB}
+  * YOUNG={YBUL, YCOW},
+  * VEGE={TOMA},
+  * SETT={SETA, NONF, FALL, GRAS},
+  * PULS=PULS
+  * PIG={PIGF, SOWS},
+  * OILS={RAPE, SOYA, SUNF, PARI, OLIV},
+  * INDS={TOBA, TEXT, TABO},
+  * GOAT={SHGM, SHGF},
+  * FRU={APPL, CITR, TAGR, TWIN},
+  * FOD={ROOF, MAIF},
+  * FLOWER={FLOW, NURS},
+  * DENNY={PORK, SOWS},
+  * COW={DCOW, SCOW, HEIF, HEIR, CAMF, CAFF, BULF, CAMR, CAFR},
+  * CHICK1={HENS, POUF},
+  * CERE={SWHE, DWHE, BARL, OATS, RYEM, MAIZ},
+  * ARAB={POTA, SUGB}
+. The sets of Northern European, Southern European countries:
+  * NEUR={NL000, UK000, AT000, BL000, DE000, DK000, FI000, FR000, SE000}
+  * SEUR={El000, ES000, PT000, IT000, IR000}
+** Table: Codes of land use classes (Set LandUse)**
+^Code  ^Label  ^
+| OART | artificial |
+| ARAO | (other) arable crops - all arable crops excluding rice and fallow (see also definition of ARAC below) |
+| PARI | paddy rice (already defined) |
+| GRAT | temporary grassland (alternative code used for CORINE data, definition identical to TGRA |
+| FRCT | fruit and citrus |
+| OLIVGR | Olive Groves |
+| VINY | vineyard (already defined) |
+| NUPC | nursery and permanent crops (Note: the aggregate PERM also includes flowers and other vegetables |
+| BLWO | board leaved wood |
+| COWO | coniferous wood |
+| MIWO | mixed wood |
+| POEU | plantations (wood) and eucalyptus |
+| SHRUNTC | shrub land - no tree cover |
+| SHRUTC | shrub land - tree cover |
+| GRANTC | Grassland - no tree cover |
+| GRATC | Grassland - tree cover |
+| FALL | fallow land (already defined) |
+| OSPA | other sparsely vegetated or bare |
+| INLW | inland waters |
+| MARW | marine waters |
+| KITC | kitchen garden |
+** Table: Codes of land use aggregates (Set LandUseAgg)**
+^Code  ^Label  ^
+| OLND | other land - shrub, sparsely vegetated or bare |
+| ARAC | arable crops |
+| FRUN | fruits, nursery and (other) permanent crops |
+| WATER | inland or marine waters |
+| ARTIF | artificial - buildings or roads |
+| OWL | other wooded land - shrub or grassland with tree cover (definition to be discussed) |
+| TWL | total wooded land - forest + other wooded land |
+| SHRU | shrub land |
+| FORE | forest (already defined) |
+| GRAS | grassland (already defined) |
+| ARAB | arable (already defined) |
+| PERM | permanent crops (already defined) |
+| UAAR | utilizable agricultural area (already defined) |
+| ARTO | total area - total land and inland waters |
+| ARTM | total area including marine waters |
+| CROP | crop area - arable and permanent |
+**Table: Codes of mutually exclusive subset adding up to total area - ARTO (Set LandUseARTO)**
+^Code  ^Label  ^
+| OLND | other land - shrub, sparsely vegetated or bare |
+| ARTIF | artificial - buildings or roads |
+| FORE | forest |
+| UAAR | utilizable agricultural area |
+| INLW | Inland waters |
+====Annex: Detailed description of Eurostat data processing in COCO (coco1_eurostat.gms)====
+The program starts by importing pre-processed data from Eurostat. The pre-processing includes simple data selection routines and also manual checks. The Eurostat domains are processed one by one, and the corrections are done for each Member State ((Eurostat offers data for Belgium and Luxembourg separately, whereas the database combines both countries to the model region "BL000" (Belgium and Luxembourg). The key reason is that Eurostat offers data mainly for the aggregate Belgium and Luxembourg up to the year 1999, especially for all market balances. Furthermore, Luxembourg has a rather small agricultural sector (2004 total output was about EUR 250 million) with some similarities to Belgium.))
+Below we discuss the specific data-processing tasks related to Eurostat table groups.
+The first Eurostat Table Group is “p_AgriProd” covering market balances and activity levels.
+//Corrections and complements for all MS://
+  * The following  data are not anymore available form Eurostat, starting with the 2010 data extractionBeginning with Eurostat selection 2010 some data are missing from the Eurostat website:
+    * DWH1, RAP1, POT1, POT2, ROO1 and ROO2 are not longer supported
+    * data for slaughter heads and slaughter tons for calves are only available for recent years
+    * deliveries to dairy of RMLK missing for earlier years in selection starting with February 2018
+For an Interim solution, data for the missing data points are collected from an earlier Eurostat selection (March 2010).
+  * UNFCCC data is included, here sheep and goats population, to prolong data of some countries where Eurostat data collection stopped 2008/2009.
+  * Recent dairy sector data from Eurostat via DG supplements the ordinary dairy data downloaded from the website of Eurostat.
+  * Sugar trade data from the market balances of Eurostat is extended with Comext (Eurostat) data.
+  * For the milk products WMIO, SMIP, FRMI and COCM some market balance positionpositions are corrected: “industrial use” is added to “feed on market and “processing” is added to “human consumption.
+  * COCO code "FRUI" is aggregated from auxiliary data for fruit trees, plus soft fruits, plus strawberries.
+  * All activities for the aggregate ILAM are added up from SHEP and GOAT.
+  * The units for wine balance sheets are converted from 1000hl to 10000hl=1000000l
+  * A rice milled equivalent balance without paddy rice (separate product) is constructed.
+  * Survey data on buffaloes are used to increase the bovine stock data to cover the whole cattle herd.
+//Corrections and complements for specific MS://
+Due to years of database updates, a number of corrections on input data are carried out. For special cases in some MS, data are read in from additional data sources:
+  * Belgium-Luxemburg: trade for potatoes (Eurostat: EU trade since 1988 by HS2-HS4 [DS-016894])
+  * France: market balances for cereal products (Agreste, Direction générale des douanes et droits indirects (DGDDI))
+  * Denmark: market balances for some cereal products (StatBank Denmark)
+  * Finland: market balances for some cereal products (Natural Resources Institute Finland, Balance sheet for food commodities)
+  * Germany: activity levels for textile crops (BMELF)
+  * Ireland: trade for citrus fruits and some milk products (Eurostat: EU trade since 1988 by HS2-HS4 [DS-016894]) and activity levels for grass land (StatBank Ireland)
+  * Austria: production of cow milk, fruit products and potatoes (Statistisches Amt Österreich)
+  * Czechia: trade of life animals (Eurostat: EU trade since 1988 by CN8 [DS-016890])
+  * Lithuania: human consumption cereal products (calculated from data from statistical yearbook 2018)
+  * Slovenia: slaughtering (SiStat Slovenia)
+  * Romania: data for the meat and in the milk sectors (Romanian experts)
+  * Trade data for sugar are collected from Eurostat COMEXT data.
+The remaining domains/table groups only require a few case-by-case corrections:
+  * The second Eurostat Table Group is “p_ExchRate” covering exchange rates
+  * The third Eurostat Table Group is “p_EcoAct” covering the economic accounts for agriculture.
+  * The fourth Eurostat Table Group is “p_AgriPri” covering agricultural producer prices.
+====Annex: Testing procedure and checking intermediate steps in COCO (biofuels)====
+The COCO module produces various reporting files on the intermediate data processing steps. These files can be used to trace back potential errors in the COCO database to their origin. These debugging files also contain meta-information on the input data and settings used for producing the COCO database.
+The following example is a walk-through on the typical data processing steps, covering biofuels data preparation in France.
+The reporting file 'output/results/coco/biof_data_with_prep/chk_biof_data_with_prep_FR000000.gdx' reports on the data preparation for biofuels for France (FR000) in COCO1. The file includes the set ‘meta_prepare_biofuel_data’, with meta-information on the recent coco1 run (e.g. creation date of file, GAMS version used).
+{{coco_biof_1.png?nolink|}}
+The set //biofCheckItems// in the same reporting .gdx file shows all biofuel items potentially filled with numbers.
+{{coco_biof_2.png?nolink|}}
+The complete list of the biofuel items in //biofCheckItems// includes codes which are additional to the CAPRI activity codes (see Annex on code lists above). The full code list includes the following items:
+| bioECere | Ethanol processed from cereals |
+| bioESuga | Ethanol processed from sugar beets |
+| bioETwin | Ethanol processed from wine |
+| bioEFrui | Ethanol processed from fruits |
+| bioEOcro | Ethanol processed from other agricultural crops |
+| bioEExog | Ethanol processed from crops not explicit in biofuel modelling (fruits, potatoes, other crops) |
+| bioARES | Biofuels processed from crops residues |
+| bioORES | Biofuels processed from forest residues and waste material (municipal waste, waste oil, other waste) |
+| SECG | Biofuel quantities from second generation |
+| MAPRagr | Ethanol production from agricultural sources |
+| EloBio | Biofuel production and demand data from DG Energy project EloBio |
+| DG_Agri | Ethanol data from DGAgri website and supplementary files |
+| ProdCom | Eurostat: PRODCOM ANNUAL SOLD (NACE Rev. 2.) [DS-066341] |
+| EIA | Independent Statistics &amp; Analysis, US Energy Information Administration |
+| comext | Eurostat: Comext |
+| Energy_bal | Eurostat: Supply, transformation, consumption - renewable energies - annual data [nrg_107a] |
+| Energy_dem | Eurostat: Supply, transformation, consumption - renewable energies - annual data [nrg_102a, nrg_1073a] |
+| final | results of the calculations |
+| ODOM | other domestic use (activity from biostock calculations |
+| INDt | Sum of model results for BIOF and INDM |
+| BIOi, INDi, DOMi | intermediate activities to save data from model initialisation for later documentation. |
+Biofuels production (levels) are calculated for biodiesel (BIOD) and bioethanol (BIOE). Input data and final initialization values before the consistency models are run are documented on the parameter //p_prepare_biofuelsMS// (see examples below). The results of the consistency models m_bioFitD (BIOD) and m_bioFitE (BIOE) are documented on the parameter //p_biofDatatMS// (see examples below).
+**Example 1: Bioethanol**
+The screenshot demonstrates the input data and final initialization values collected on parameter //p_prepare_biofuelsMS//. The first column of the table indicates the data source, respectively the processing status of the data. Data sources for bioethanol (BIOE) include data from EloBio, DG_Agri, ProdCom, EIA, Engergy_bal and Energy_dem. The second column of the table shows the activity.
+{{coco_biof_3.png?nolink|}}
+The results of the model m_bioFitE (BIOE) are documented on the parameter p_biofDatatMS.
+{{coco_biof_4.png?nolink|}}
+We take soft wheat (SWHE) as an example for biofuel feedstock, and walk through the initialization and consistency model results. From data input (Eurostat and FAO) we received in 2002 an industrial use of 894 1000t, saved on INDi. For production of bio-ethanol 631 1000t were initialized, saved on BIOi. The results of the breakdown by use for bio-ethanol and others industrial use, are saved on BIOF and INDM. BIOE shows the yield of soft wheat for bio-ethanol.
+**Example 2: Biodiesel**
+The first dimension of the reporting parameter //p_prepare_biofuels// shows the data source (processing status).
+The second dimension of the parameter shows the activity.
+{{coco_biof_5.png?nolink|}}
+For Bio-diesels, PRIMES model results are used as an additional data source.
+^Data source code ^Data source description ^
+|Primes		|PRIMES MODEL, EC3MLAB of ICCS, National University of Athens|
+{{coco_biof_6.png?nolink|}}
+The parameter //p_biofDataMS// reports on production (MAPR), trade (import:IMPT, export:EXPT), production from non-agricultural sources (NAGR), prices (UVAD, UVAP) and consumer taxes (CTAX). The distribiution of total biodiesel processing to the feedstock is also reported, for rapeseed oil (RAPO), sunflower oil (SUNO), soya oil (SOYO) and palm oil (PLMO).
+====Annex: Testing procedure and checking intermediate steps in COCO (dairy)====
+The following three examples show how to use the intermediate reporting files to trace the data preparation steps. Screenshots demonstrate the arrangement of the reporting parameters by using the CAPRI Graphical User Interface. COCO automatically produces the reporting files in the folder // results/coco/res_estima/ //
+**Example 1: Production of cow (COMI) and sheep (SGMI) milk**
+In order to document the procedure of data consolidation and rebooking, we look at the reporting file for France “chk_estima_FR000.gdx”.
+{{:wiki:coco_dairy_1.png?nolink|}}
+The codes in the rows show the activity code, the product code and its status. For activity codes see Annex 1: Code list.
+Status codes:
+  * INI:	initial value
+  * COCO1:	estimation value
+The initialization of the production of COMI and SGMI is done in the module //coco1_milk.gms// (see section 3.1.3). Additional remarks to better understand the example:
+  * COMI: Milk from cows (CMLK) and buffaloes (BMLK) is added up.
+  * SGMI: Milk from ewes (EMLK) and goats (GMLK) is added up.
+  * If data on cow or sheep and goat milk is not available separately, but total milk production (RMLK) is available, then production of COMI is set equal to total milk production.
+  * Only COMI and SGMI are included in the estimation in //coco1_estima.gms//
+  * The production of RMLK and its components CMLK, BMLK, EMLK and GMLK are only copied from raw data tables into this check parameter for documentation purposes.
+** Example 2: data consolidation for cow milk**
+The procedure of data consolidation and rebooking of all activities for the CAPRI product “COMI” (cow milk) is shown in the following screenshot (only part of the reporting parameter p_estimAnimMS is shown, but the full scope of the table is visible in the GUI).
+{{:wiki:coco_dairy_2.png?nolink|}}
+The codes in the rows show the activity code and its status. For activity codes see Annex 1: Code list.
+Additional codes for status include the following.
+^Status code   ^Status code description  ^
+|StdeData	|Final (small) Stde (standard deviation) attached to priors from raw data |
+|StdeScale	|Final (large) Stde attached to priors from trends but not from raw data  |
+|Upplim		|Soft upper limits triggering extra penalties if violated                 |
+|Lowlim		|Soft lower limits triggering extra penalties if violated                 |
+|Supps		|Prior value = support: comes from raw data or trends plus HP filter      |
+|Err2rev	|Original error term from preest: to steer speed of bound opening         |
+Under activity dairy cows (DCOW) the following items are reported: yield, total production (GROF), feed use (FEDM) and losses on market (LOSM). Eurostat’s //National Accounts of Agriculture (EAA)// only supply data for the aggregate milk (MILK). The equation //e_EAAMLK// in the consolidation model //AnimNSSQ// ensures the consistency of EAA values for MILK, as they are split up between cow milk (COMI) and sheep and goat milk (SGMI).
+<code fortran>
+e_EAAMLK("%MS%000",T)
+     $ (p_NobsP("%MS%000","EAAP","MILK") AND ESTR("MILK") and
+        (p_NobsP("%MS%000","EAAP","COMI") or p_NobsP("%MS%000","EAAP","SGMI"))) ..
+*
+        v_EstimY("%MS%000","EAAP","MILK",T) =E=
+                             v_EstimY("%MS%000","EAAP","COMI",T) $ p_NobsP("%MS%000","EAAP","COMI")
+                           + v_EstimY("%MS%000","EAAP","SGMI",T) $ p_NobsP("%MS%000","EAAP","SGMI") ;
+</code>
+Finally. the producer prices (UVAP) are calculated directly from the monetary EEA values and production.
+The following picture shows the data processing steps (states) for the EAA values for milk.
+{{:wiki:coco_dairy_3.png?nolink|}}
+From the example for COMI above you can also understand the influence of the standard deviation from raw data (e.g. FEDM.StdeData), and standard deviation from trends (e.g. FEDM.StdeScale) Standard deviations are calculated both for raw data and the trends. For years where FEDM.StdeData is given, the results are very close to the prior values FEDM.Supps, whereas they are deviating sizeably for years where only FEDM.StdeScale is available.
+The first initialisation of //StdeData// and //StdeScale// is done in module coco1_preest.gms, which is a pre-step for the data consolidation models (crops, animals, market balances), using a Hodrick-Prescott filter to smooth the combination of given values and trend line. Both standard deviations enter the objective function (see chapter 3.1.4).
+**Example 3: data consolidation for cow dairy cow activity (DCOW)**
+The procedure of data consolidation and booking intermediate data processing results for the dairy cow activity (DCOW) is demonstrated in the following screenshot.
+{{:wiki:coco_dairy_4.png?nolink|}}
+The rows of the table show the product item code for the production activity DCOW, and the data processing steps (status). The first two lines show the coco1 results for slaughtering. The items starting with Y and I stand for the output and input of calves. The initialization, the estimation steps and the final results are all documented on the reporting parameter //p_estimAnimMS//. Items COMI and BEEF show the yields for cow milk and beef. Item DAYS is the process length, initialized by 365 days (equals one year). Finally, the item HERD models the herd size of dairy cows.
 ===== The Regionalised Data Base (CAPREG) =====
@@ Line 1046: / Line 1551: @@
 | Agricultural accounts on regional level | from 1980 yearly |
 | Structure of agricultural holdings and labour force | 2000, 2003, 2005, 2007, 2010, 2013 |
-<sup> Source: capri\dat\capreg\regio_data_all.gdx </sup>
+<sup> Source: capri/dat/capreg/regio_data_all.gdx </sup>
 ====Methodology applied in the regional data consolidation====
 In the last major update of 2015 the original data had been first stored in the TSV format designed by EUROSTAT:
   * Unordered List ItemIn a first step, these files had been converted by an excel macro into csv format and an overall set with all items including their long text has been created to prepare further processing.
-  * In a second step these alredy GAMS readable files are stored in GDX format in folder “dat\capreg” and under version control. Meta data are added in the process as well.
+  * In a second step these alredy GAMS readable files are stored in GDX format in folder “dat/capreg” and under version control. Meta data are added in the process as well.
@@ Line 1060: / Line 1565: @@
 Given that starting position, the following approaches are generally applied:
-  * Unordered List ItemData as loaded from the regional statistics are subject to some manual consistency checks (in gams\capreg\check_and_cor_regio.gms) as well as checks for regional consistency. The latter is mainly true for animal herd sizes where we have data at the same or even more disaggregated level as found in COCO.
+  * Unordered List ItemData as loaded from the regional statistics are subject to some manual consistency checks (in gams/capreg/check_and_cor_regio.gms) as well as checks for regional consistency. The latter is mainly true for animal herd sizes where we have data at the same or even more disaggregated level as found in COCO.
   * Gaps in regional data are completed and data only given at a higher aggregation level as required in CAPRI are broken down by using existing national information.
   * Fall back and other rules for assignments are structurally and (often) numerically identical for all regional units and groups of activities and inputs/outputs.
@@ Line 1079: / Line 1584: @@
 Young animals are valued based on the ‘meat value’ and assumed relationships between live and carcass weights. Male calves (ICAM, YCAM) are assumed to have a final weight of 55 kg, of which 60 % are valued at veal prices. Female calves (ICAF, YCAF) are assumed to have a final weight of 60 kg, of which 60 % are valued at veal prices. Young heifers (IHEI, YHEI) are assumed to have a final weight of 300 kg, of which 54 % are valued at beef. Young bulls (IBUL, YBUL) are assumed to have a final weight of 335 kg, of which 54 % are valued at beef. Young cows (ICOW, YCOW) are assumed to have a final weight of 575 kg, of which 54 % are valued at beef. For piglets (IPIG, YPIG), price notations were regressed on pig meat prices and are assumed to have a final weight of 20 kg of which 78 % are valued at pig meat prices. Lambs (ILAM, YLAM) are assumed to weight 4 kg and are valued at 80 % of sheep and goat meat prices. Chicken (ICHI, YCHI) are assumed to weight 0.1 kg and are valued at 80 % of poultry prices.
-Another special case are sugar beet prices. They are still determined in a program (//‘sugar\price_est.gms’//) inherited from the 2003 EuroCARE sugar study (Henrichsmeyer et al. 2003). It determines sugar beet prices according to the sugar prices, levies and partial survey results in the 90ies. The estimation results are subsequently used to determine the beet price differentiation also in subsequent years. It is noteworthy that the same program is applied in CAPREG (via quotasprices.gms) and in CAPMOD (via data_prep.gms) to determine base year beet prices.
+Another special case are sugar beet prices. They are still determined in a program (//‘sugar/price_est.gms’//) inherited from the 2003 EuroCARE sugar study (Henrichsmeyer et al. 2003). It determines sugar beet prices according to the sugar prices, levies and partial survey results in the 90ies. The estimation results are subsequently used to determine the beet price differentiation also in subsequent years. It is noteworthy that the same program is applied in CAPREG (via quotasprices.gms) and in CAPMOD (via data_prep.gms) to determine base year beet prices.
 === Activity Levels===
@@ Line 1103: / Line 1608: @@
 One important advantage of the approach is the fact that the resulting areas are automatically consistent to the national data if the ingoing information from REGIO was consistent to national level. Fortunately, the regional information on herd sizes covers most of the data needed to give nice proxies for all animal activities in COCO definition. The regional data break down for herd sizes is often more detailed than COCO  at least for the important sectors. Regional estimates for the activity levels are therefore the result of an aggregation approach, in opposite to crop production.
-In order to generate good starting points for the following steps of data processing and to avoid systematic deviations between regional and national levels in the following consistency steps, all regional level in REGIO are first scaled with the relation between the (national) results in COCO and the regional results when aggregated to the national level (key file is gams\capreg\map_from_regio.gms).
+In order to generate good starting points for the following steps of data processing and to avoid systematic deviations between regional and national levels in the following consistency steps, all regional level in REGIO are first scaled with the relation between the (national) results in COCO and the regional results when aggregated to the national level (key file is gams/capreg/map_from_regio.gms).
-Besides technological plausibility and a good match with existing regional statistics, the regionalized data for the CAPRI model must be also consistent to the national level. The minimum requirement for this consistency includes activity levels and gross production. The “initialisation” of the regional database has been undertaken already to meet this requirement as good as possble but cannot guarantee it. Consistency for activity levels is therefore based on Highest Posterior Density Estimator which ensures (in gams\capreg\cons_levls.gms):
+Besides technological plausibility and a good match with existing regional statistics, the regionalized data for the CAPRI model must be also consistent to the national level. The minimum requirement for this consistency includes activity levels and gross production. The “initialisation” of the regional database has been undertaken already to meet this requirement as good as possble but cannot guarantee it. Consistency for activity levels is therefore based on Highest Posterior Density Estimator which ensures (in gams/capreg/cons_levls.gms):
   - Adding up of activity levels from lower regional level (NUTS II, NUTS I) to higher ones (NUTS I, NUTS 0)
@@ Line 1112: / Line 1617: @@
 The objective function minimizes in case of animal herds simple squared relative deviations from the herds. In case of crops, a 25% weight for absolute squared difference of the crop shares on UAA plus 75% deviation of relative squared differences is introduced. In the crop sector consistency is also imposed to regional transition matrices for 6 UNFCCC land use categories relevant for carbon accounting (forest land, cropland, grassland, settlements, wetlands, residual land) which are initialised from the national transition matrix estimated in the COCO1 module.
-A specific problem is the fact that land use statistics do not report a break down of idling land into obligatory set aside, voluntary set aside and fallow land((The necessary additional information on non-food production on set-aside, obligatory and voluntary set-aside areas can be found on the DG-AGRI web server.)). Equally, the share of oilseeds grown as energy crops on set aside needs to be determined. An Highest Posterior density estimator is used (in gams\capreg\cal_seta.gms) to ‘distribute’ the national information on the different types of idling land to regional level, with the following restrictions:
+A specific problem is the fact that land use statistics do not report a break down of idling land into obligatory set aside, voluntary set aside and fallow land((The necessary additional information on non-food production on set-aside, obligatory and voluntary set-aside areas can be found on the DG-AGRI web server.)). Equally, the share of oilseeds grown as energy crops on set aside needs to be determined. An Highest Posterior density estimator is used (in gams/capreg/cal_seta.gms) to ‘distribute’ the national information on the different types of idling land to regional level, with the following restrictions:
   * Obligatory set-aside areas must be equal to the set-aside obligations derived from areas and set-aside rates for Grandes Cultures (which may differ at regional level according to the share of small producers). For these crops, activity levels are partially endogenous in the estimation in order to allow a split up of oilseeds into those grown under the set-aside obligations and those grown as non-fo-od crops on set-aside.
   * Obligatory and voluntary set-aside cannot exceed certain shares of crops subjects to set-aside (at least before Agenda 2000 policy)
@@ Line 1145: / Line 1650: @@
 The regional database modules also cover some aspects which are discussed in other parts of this documentation.
-  * For policy data at the regional level (mostly premium related data) see Section [[Policy data]]. These policy related assignments require a good part of the CAPREG module
+  * For policy data at the regional level (mostly premium related data) see Section [[the capri data base#Policy data]]. These policy related assignments require a good part of the CAPREG module
-  * For the fertiliser and feed allocations and environmental indicators, also important elements of the regional database, see the next Section [[Input Allocation]]
+  * For the fertiliser and feed allocations and environmental indicators, also important elements of the regional database, see the next Section [[the capri data base#Input Allocation]]
-  * Towards the end of the regional data base consolidation supply side PMP parameters are calibrated as a final test of consistency and sometimes to serve as starting values for the subsequent baseline calibration (in //gams\capreg\pmp.gms//)
+  * Towards the end of the regional data base consolidation supply side PMP parameters are calibrated as a final test of consistency and sometimes to serve as starting values for the subsequent baseline calibration (in //gams/capreg/pmp.gms//)
 ====Build and compare time series of GHG inventories====
@@ Line 1155: / Line 1660: @@
   * The second step involves more time consuming processing steps which are therefore only executed for the selected base year: feed allocation, computation of GHG results, and the final calibration test
-To assess the reliability of the CAPRI database in terms of GHG results against official UNFCCC notifications, results from the first step (time series) were insufficient, as the GHG accounting also requires information on the feed allocation. This problem was addressed within the scope of the IDEAg (Improving the quantification of GHG emissions and flows of reactive nitrogen) project((The IDEAg project was commissioned by the JRC-IES in Ispra in 2015 and was carried out by the Thünen Institute in cooperation with the JRC-IES (August 2015 – August 2016). A more detailed explanation of the CAPRI task “Build GHG inventories” and its use has been prepared by the Thünen contributors at the time, Sandra Marquardt and Alexander Gocht, see capri\doc\GHG_inventory_module.docx. )), where an option has been introduced to allow for a consistent accounting of GHG emissions over time. This is able to combine input information from CAPREG time series runs as well as (short run, nowcasting-style) CAPMOD simulation results. Furthermore, an R-based tool was introduced to the CAPRI GUI that maps GHG emissions data from CAPRI to the GHG emission balances contained in the National Inventory Reports (NIRs) that are submitted annually by countries in compliance with UNFCCC GHG reporting obligations.
+To assess the reliability of the CAPRI database in terms of GHG results against official UNFCCC notifications, results from the first step (time series) were insufficient, as the GHG accounting also requires information on the feed allocation. This problem was addressed within the scope of the IDEAg (Improving the quantification of GHG emissions and flows of reactive nitrogen) project((The IDEAg project was commissioned by the JRC-IES in Ispra in 2015 and was carried out by the Thünen Institute in cooperation with the JRC-IES (August 2015 – August 2016). A more detailed explanation of the CAPRI task “Build GHG inventories” and its use has been prepared by the Thünen contributors at the time, Sandra Marquardt and Alexander Gocht, see capri/doc/GHG_inventory_module.docx. )), where an option has been introduced to allow for a consistent accounting of GHG emissions over time. This is able to combine input information from CAPREG time series runs as well as (short run, nowcasting-style) CAPMOD simulation results. Furthermore, an R-based tool was introduced to the CAPRI GUI that maps GHG emissions data from CAPRI to the GHG emission balances contained in the National Inventory Reports (NIRs) that are submitted annually by countries in compliance with UNFCCC GHG reporting obligations.
 ===== Input Allocation =====
@@ Line 1176: / Line 1681: @@
 Given the importance or the input allocation, the CAP STRAT project (2000-2003) comprised an own work package to estimate input coefficients. On a first step, input coefficients were estimated using standard econometrics from single farm record as found in FADN. Additionally, tests for a more complex estimation framework building upon entropy techniques and integrating restrictions derived from cost minimization were run in parallel. The need to accommodate the estimation results with data from the EAA in order to ensure mutual compatibility between income indicators and input demand per activity and region on the one hand, and sectoral income indicators as well as sectoral input use on the other, requires deviating from the estimated mean of the coefficients estimated from single farm records. Further on, in some cases estimates revealed zero or negative input coefficients, which cannot be taken over. Accordingly, it was decided to set up a second stage estimation framework building upon the unrestricted estimates from FADN. The framework can be applied to years where no FADN data are available, and thus ensures that the results will be continuously used for the years ahead, before an update of the labor-intensive estimations is again necessary and feasible.
-As a result of the unrestricted estimation based on FADN ((More details on the FADN estimation were reported in older versions of the CAPRI documentation, accessible in the \doc folder of any stable release of the CAPRI system up to star 2.4 from [[https://www.capri-model.org/dokuwiki/doku.php?id=capri:get-capri]].))a matrix of input coefficients for 11 input categories (Total Inputs, Crop Only Inputs, Animal Only Inputs, Seeds, Plant Protection, Fertilizer, Other Crop Inputs, Purchased and Non-Purchased Feeds and Other Animal Only Inputs) and their estimated standard errors is available. Some of those coefficients are related to the output of a certain activity (e.g. how much money is spend on a certain input to produce one unit of a product), some of them are related to the acreage of on activity (input costs per activity level).
+As a result of the unrestricted estimation based on FADN ((More details on the FADN estimation were reported in older versions of the CAPRI documentation, accessible in the /doc folder of any stable release of the CAPRI system up to star 2.4 from [[https://www.capri-model.org/dokuwiki/doku.php?id=capri:get-capri]].))a matrix of input coefficients for 11 input categories (Total Inputs, Crop Only Inputs, Animal Only Inputs, Seeds, Plant Protection, Fertilizer, Other Crop Inputs, Purchased and Non-Purchased Feeds and Other Animal Only Inputs) and their estimated standard errors is available. Some of those coefficients are related to the output of a certain activity (e.g. how much money is spend on a certain input to produce one unit of a product), some of them are related to the acreage of on activity (input costs per activity level).
 All of the econometric coefficients were required to be transformed into an ‘activity level’ form, due to the fact that this is the definition used in the CAPRI model. Before this could be done, it seemed necessary to fill up the matrix of estimated coefficients because some estimates were missing and others were negative. In order to this we constructed a number of coefficients that were weighted averages among certain groups. These mean coefficients were the following.
@@ Line 1185: / Line 1690: @@
 Following these rules we finally got a matrix of estimated and synthetic calculated input coefficients for both, the ‘per activity level’ and the ‘per production’ unit definition((In addition, a similar procedure (using slightly different groups) was applied to constructing coefficients for the ‘Other’ activities (e.g. OCER, OFRU, OVEG), which had been omitted from the econometric estimations. They are given the average group coefficient, unless there is none; then they are given the average northern or southern European coefficient as appropriate.)). For the synthetic one there was no estimated standard error available but we wanted to use those later on. So we assumed them –to reflect that these coefficients have only weak foundation– to have a t statistic of 0.5.
-The ‘per level’ definition was only taken over if the coefficient was really estimated or if no per production unit definition did exist. To transfer the latter into per activity level definition, we multiplied them with the average yield (1985 2001) of the respective activity. The resulting coefficients and their standard errors were then used a HPD approach as a //first set of priors//((The previously described completions are implemented in file gams\input\fill_inp_matrix.gms. Adjustments were made for scaling issues with regard to eggs for certain countries, and grass for Finland. In addition, when ‘CAFR’,’CAFF’ and ‘HEIR’ did not have econometric data, they assumed the coefficients and standard errors of ‘CAMR’, ‘CAMF’ and ‘HEIF’ respectively (CAPRI activity code definitions in the Annex).)).
+The ‘per level’ definition was only taken over if the coefficient was really estimated or if no per production unit definition did exist. To transfer the latter into per activity level definition, we multiplied them with the average yield (1985 2001) of the respective activity. The resulting coefficients and their standard errors were then used a HPD approach as a //first set of priors//((The previously described completions are implemented in file gams/input/fill_inp_matrix.gms. Adjustments were made for scaling issues with regard to eggs for certain countries, and grass for Finland. In addition, when ‘CAFR’,’CAFF’ and ‘HEIR’ did not have econometric data, they assumed the coefficients and standard errors of ‘CAMR’, ‘CAMF’ and ‘HEIF’ respectively (CAPRI activity code definitions in the Annex).)).
 Missing econometric estimates and compatibility with EAA figures were not the only reasons that made a reconciliation of estimated inputs coefficients necessary. Moreover, the economic sense of the estimates could not be guaranteed and the definition of inputs in the estimation differed from the one used in CAPRI. Therefore we decided to include further prior information on input coefficients in agriculture. The //second set of priors// in the input reconciliation was therefore based on data from the EAA. Total costs of a certain input within an activity in a European Member State was calculated by multiplying the total expenditures on that input with the proportion of the total expected revenue of that activity to that of all activities using the input. Total expected revenue in this case was the production value (including market value and premiums) of the respective activity. If this resulted in a certain coefficient being calculated as zero due to missing data, then this coefficient would be replaced by one from a similar activity e.g. a zero coefficient for ‘MAIF’ would be replaced by the coefficient for ‘GRAS’
-This kind of prior information tries to give the results a kind of economic sense. For the same reason the //third type of priors// was created based on standard gross margins for agricultural activities received from EUROSTAT. Those existed for nearly all activities. The set from 1994 was used, since this was the most complete available. Relative rather than absolute differences were important, given the requirement to conform to EAA values((Contrary to the econometric estimated priors, the two other types were different in different years, since the reconciliation had to be done for each year in the database. The second prior type is year specific by nature, as the EAA values differ between years. In case of standard gross margins, unfortunately, we had them only for one year (1994). So we decided to ‘drive them over time’ using the proportion of expected revenue of an activity in a certain year to that in the year 1994. Furthermore it may be mentioned that for plant protection coefficients a fourth set of priors from an industry source has been used and that energy inputs also received a special treatment in the key file gams\input\dist_inputs.gms.)).
+This kind of prior information tries to give the results a kind of economic sense. For the same reason the //third type of priors// was created based on standard gross margins for agricultural activities received from EUROSTAT. Those existed for nearly all activities. The set from 1994 was used, since this was the most complete available. Relative rather than absolute differences were important, given the requirement to conform to EAA values((Contrary to the econometric estimated priors, the two other types were different in different years, since the reconciliation had to be done for each year in the database. The second prior type is year specific by nature, as the EAA values differ between years. In case of standard gross margins, unfortunately, we had them only for one year (1994). So we decided to ‘drive them over time’ using the proportion of expected revenue of an activity in a certain year to that in the year 1994. Furthermore it may be mentioned that for plant protection coefficients a fourth set of priors from an industry source has been used and that energy inputs also received a special treatment in the key file gams/input/dist_inputs.gms.)).
 Given the three types of prior information explained above –estimated input coefficients, data from EAA and standard gross margins , a HPD estimator has been used to reconcile the prior information on input coefficients. Accounting constraints ensure (see in “dist_input.gms”) first that gross margins for an activity is the difference between expected revenue per activity level of that activity and the sum over all inputs used in that activity and second that the sum over all activities of their activity levels multiplied with an input gives the total expenditures on that input given by the EAA. The estimation is carried out in GAMS within and run for each year in the database. Some bounds are further set to avoid estimates running into implausible ranges.
@@ Line 1197: / Line 1702: @@
 ====Input allocation for young animals and the herd flow model ====
-Figure below shows the different cattle activities and the related young animal products used in the model. Milk cows (DCOL, DCOH) and suckler cows (SCOW) produce male and female calves (YCAM, YCAF). The relation between male and female calves is estimated ex post in the COCO framework. These calves are assumed to weigh 50 kg at birth (see gams\feed\feed_decl.gms) and to be born on the 1st of January. They enter immediately the raising processes for male and female calves (CAMR, CAFR) which produce young heifers (YHEI, 300 kg live weight) and young bulls (YBUL, 335 kg). The raising processing are assumed to take one year, so that calves born in t enter the processes for male adult fattening (BULL, BULH), heifers fattening (HEIL, HEIH) or heifers raising (HEIR) on the 1st January of the next year t+1. The heifers raising process produces then the young cows which can be used for replacement or herd size increasing on the first of January of t+2. The table below the diagram shows a numerical example (for DK, 1999-2001) for these relationships.
+Figure below shows the different cattle activities and the related young animal products used in the model. Milk cows (DCOL, DCOH) and suckler cows (SCOW) produce male and female calves (YCAM, YCAF). The relation between male and female calves is estimated ex post in the COCO framework. These calves are assumed to weigh 50 kg at birth (see gams/feed/feed_decl.gms) and to be born on the 1st of January. They enter immediately the raising processes for male and female calves (CAMR, CAFR) which produce young heifers (YHEI, 300 kg live weight) and young bulls (YBUL, 335 kg). The raising processing are assumed to take one year, so that calves born in t enter the processes for male adult fattening (BULL, BULH), heifers fattening (HEIL, HEIH) or heifers raising (HEIR) on the 1st January of the next year t+1. The heifers raising process produces then the young cows which can be used for replacement or herd size increasing on the first of January of t+2. The table below the diagram shows a numerical example (for DK, 1999-2001) for these relationships.
 **Figure 5: The cattle chain**
-{{:figure_5.png?600|}} \\ Source: CAPRI Modelling System
+{{:figure_5.png?600|Source: CAPRI Modelling System}}
 Accordingly, each raising and fattening process takes exactly one young animal on the input side. The raising processes produce exactly one animal on the output side which is one year older. The output of calves per cow, piglets per sow, lambs per mother sheep or mother goat is derived ex post, e.g. simultaneously from the number of cows in t-1, the number of slaughtered bulls and heifers and replaced in t+1 which determine the level of the raising processes in t and number of slaughtered calves in t. The herd flow models for pig, sheep and goat and poultry are similar, but less complex, as all interactions happen in the same year, and no specific raising processes are introduced.
@@ Line 1262: / Line 1767: @@
 |//Sum of slaughtered cows and stock change//|		|	|235,45|
 |GROFYCOW|	Numer of heifers raised to young cows|	235,45	|227,16	|229,4|
-|HEIRLEVL|	Activity level of the heifers raising process	|235,45	|227,16	|229,4|
+|HEIRLEVL|	Activity level of the heifers raising process	|235,45	|227,16	|229,4| \\ Source: CAPRI Modelling System
- \\ Source: CAPRI Modelling System
@@ Line 1273: / Line 1777: @@
 |Dairy cows (DCOW)	|DCOL: 60% milk yield of average, variable inputs besides feed an young animals at 60% of average	|DCOH: 140% milk yield of average, variable inputs besides feed an young animals at 140% of average|
 |Bull fattening (BULF)	|BULL: 20% lower meat output, variable inputs besides feed an young animals at 80% of average	|BULH: 20% higher meat output, variable inputs besides feed an young animals at 120% of average|
-|Heifers fattening (HEIF)|	HEIL: 20% lower meat output, variable inputs besides feed an young animals at 80% of average	|HEIH: 20% higher meat output, variable inputs besides feed an young animals at 120% of average|
+|Heifers fattening (HEIF)|	HEIL: 20% lower meat output, variable inputs besides feed an young animals at 80% of average	|HEIH: 20% higher meat output, variable inputs besides feed an young animals at 120% of average| \\ Source: CAPRI Modelling System
- \\ Source: CAPRI Modelling System
 ====Input allocation for feed====
@@ Line 1372: / Line 1875: @@
 __ FeedAggrShare_ __
-{{:code_p_72_2.png?600|}}
+{{:code_p_72_2.png?600}}
 __ MeanFeedTotal_ __
-{{:code_p_72_3.png?600|}}
+{{:code_p_72_3.png?600}}
 One of the aggregates calculated is the total feed intake per animal. It is expected that, inspite of regional differences in fodder supply, this total feed intake is mostly a genetic characteristic of animals and hence should not vary markedly across regions. To influence this distribution in the objective, the average across regions needs to be computed.
@@ Line 1434: / Line 1937: @@
 __Shares of feed aggregates in total feed intake in DRMA __
-{{::code_p_74.png?600|}}
+{{::code_p_74.png?600}}
 The shares of roughage and concentrate feed are only controlled by upper (p_maxFeedShare) and lower (p_minFeedShare) limits. The literature suggests that ruminants can digest at most 40% of concentrate feed (or at least 60% roughage), and perhaps 45% for activity DCOH. The upper and lower limits are partially taken from IFM-CAP, literature and expert knowledge of Markus Kempen (Assumed values in table 12).
@@ Line 1464: / Line 1967: @@
 __ Feed input coefficients for single feed bulks __
-{{::code_p_75.png?600|}}
+{{::code_p_75.png?600}}
 Apart from plausibility of the results a second objective of the revision has been reproducability. The previous specification essentially gave random results within the feasible set because no prior expectations had been specified. This has been revised with penalties for deviations of feed input coefficients from their assumed MEAN (specification to be explained below). However, just like is the case for the nutrient content of feed aggregates or their shares in the total, this prior information has to be considered quite imprecise which is reflected in rather low factors (1E2) attached to these terms. The penalties are increased if the solver tries to approach or exceed “soft” lower or upper limits. As the lower limits also turned out useful to prevent the solver from ending up in infeasible corners a higher factor has been attached to them (1E5).
@@ Line 1470: / Line 1973: @@
 It should also be reported that in many cases of infeasible solutions encountered in the extensive testing of this and previous specifications the last iteration result reported from the solver had often all feed input coefficients for some animal type zero or near zero. To avoid these cases the solution attempt starts with hard lower bounds:
-{{:cope_p_76.png?600|}}
+{{:cope_p_76.png?600}}
 In case of infeasibilities after x trials these are removed:
-{{:code_p_76_2.png?600|}}
+{{:code_p_76_2.png?600}}
 This procedure led to an acceptable or at least considerably improved stability of the feed calibration in tasks “build regional database” as well as “baseline calibration supply models”.
@@ Line 1480: / Line 1983: @@
 **Priors for feed input coefficients**
-The priors for feed input coefficients are specified in a new include file capri\gams\feed\fedtrm_prior.gms:
+The priors for feed input coefficients are specified in a new include file capri/gams/feed/fedtrm_prior.gms:
-{{:code_p_76_3.png?600|}}
+{{:code_p_76_3.png?600}}
 The shares of feed aggregates in the diets of animal types may build upon recommendations from the literature (see the previous section). They are adjusted to be in line with the statistical ex post data or the baseline projections, giving the “adjusted” aggregate feed input coefficients shown in the code snippet above.
@@ Line 1650: / Line 2153: @@
 **Figure 6. Ex-post calibration of NPK balances and the ammonia module**
-{{::figure_6.png?600|}} \\ Source: CAPRI modelling system
+{{::figure_6.png?600|Source: CAPRI modelling system}}
 The following equations comprise together the cross-entropy estimator for the NPK (Fnut=N, P or K) balancing problem. Firstly, the purchases (NETTRD) of anorganic fertiliser for the regions must add up to the given inorganic fertiliser purchases at Member State level:
@@ Line 1702: / Line 2205: @@
 |  **Ex-post**  |  **Ex-ante**  |
-|**Given:**\\ -Herd sizes\\  => Manure output\\ -Crop areas and yields\\  => Export with harvest\\ -National anorganic application\\ **Estimated:**\\ -Regional anorganic application\\ -Factor for Fertilization beyond N export\\ -Manure availability |**Model result:**\\ -Herd sizes\\  => manure output\\ -Crop areas and yields\\ => Export with harvest\\ -National and Regional anorganic application\\ **Given:** \\ -Factor for Fertilization beyond export (trended)\\ -Manure availability (trended)|
+|**Given:**|**Model result:**|
+|-Herd sizes|-Herd sizes|
+| => Manure output|=> manure output|
+|-Crop areas and yields|-Crop areas and yields|
+|=> Export with harvest|=> Export with harvest|
+|-National anorganic application|-National and Regional anorganic application|
+|**Estimated:**|**Given:**|
+|-Regional anorganic application|-Factor for Fertilization beyond export (trended)|
+|-Factor for Fertilization beyond N export|-Manure availability (trended)|
+|-Manure availability| |
 A good overview on how the Nitrogen balances are constructed and can be used for analysis can be found in: Leip A., Britz W., de Vries W. and Weiss F. (2011): Farm, land, and soil nitrogen budgets for agriculture in Europe calculated with CAPRI, Environmental Pollution 159(11), 3243-3253 and Leip, A., Weiss, F. and Britz, W. (2011): Agri-Environmental Nitrogen Indicators for EU27, in: Flichman G. (ed.), Bio-Economic Models applied to Agricultural Systems, p. 109-124, Springer, Netherlands.
@@ Line 1716: / Line 2228: @@
 **Figure 7: Ammonia sinks in the Ammonia emission module**
-{{::figure_7.png?600|}} \\ Source: CAPRI modelling system
+{{::figure_7.png?600|Source: CAPRI modelling system }}
 In the figure above, white arrows represent ammonia losses and are based on uniform or Member State specific coefficients. A first Member State specific coefficient characterises for each animal type the share of time spent on grassland and spent in the stable. For dairy cows, for example, the factors are between 41 % spent in the stable in Ireland and 93 % in Switzerland. During grazing about 8% of the excreted N is assumed lost as ammonia.
@@ Line 1764: / Line 2276: @@
 **Figure 8: Carbon flows in the agricultural production process**
-{{:figure_8.png?600|}} \\ Source: Weiss and Leip (2016)
+{{:figure_8.png?600|Source: Weiss and Leip (2016)}}
 In the following, we briefly describe the general methodology for the quantification of the carbon flows that are taken into account in the CAPRI approach.
@@ Line 1872: / Line 2384: @@
 |:::	|Manure management|CH4Man|
 |:::	|Rice production|CH4Ric|
-|:::	|Land use change emissions from\\ biomass burning|CH4bur|
+|:::	|Land use change emissions from biomass burning|CH4bur|
 |**Nitrous Oxide**|Manure management|N2OMan|
 |:::	|Manure excretion on grazings|N2OGra|
@@ Line 1878: / Line 2390: @@
 |:::	|Application of manure|N2OApp|
 |:::	|Crop residues|N2OCro|
-|:::	|Indirect emissions from ammonia \\ losses|N2OAmm|
+|:::	|Indirect emissions from ammonia losses|N2OAmm|
-|:::	|Indirect emissions from leaching \\ and runoff|N2OLea|
+|:::	|Indirect emissions from leaching and runoff|N2OLea|
 |:::	|Cultivation of histosols|N2Ohis|
-|:::	|Land use change emissions from the \\ burning of biomass|N2Obur|
+|:::	|Land use change emissions from the burning of biomass|N2Obur|
 |**Carbon dioxide**|Cultivation of histosols|CO2his|
 |:::	|Applicaton of ureum|CO2urea|
 |:::	|Liming|CO2lim|
-|:::	|Land use change emissions from above \\ and below ground biomass|CO2bio|
+|:::	|Land use change emissions from above and below ground biomass|CO2bio|
-|:::	|Land use change emissions from soil \\ carbon changes|CO2soi| \\ Source: CAPRI Modelling System
+|:::	|Land use change emissions from soil carbon changes|CO2soi| \\ Source: CAPRI Modelling System
 For a detailed analysis of these single emission sources refer to Pérez 2006: Greenhouse Gases: Inventories, Abatement Costs and Markets for Emission Permits in European Agriculture -A Modelling Approach and Leip et al 2010: Evaluation of livestock sector’s contribution to the RU greenhouse gas emissions (GGELS).
@@ Line 1939: / Line 2451: @@
 ==== Input allocation for labour ====
-Labour (and other inputs) in CAPRI are estimated from a Farm Accounting Data Network (FADN) sample ((More details on the FADN estimation were reported older versions of this section (originally drafted by Markus Kempen and Eoghan Garvey) the CAPRI documentation, accessible in the \doc folder of any stable release of the CAPRI system up to star 2.4 from [[https://www.capri-model.org/dokuwiki/doku.php?id=capri:get-capri.]])) and then these estimation results are combined with total labour requirements within a region (or aggregate national input demand reported in the EAA), using a Highest Posterior Density (HPD) estimation framework.
+Labour (and other inputs) in CAPRI are estimated from a Farm Accounting Data Network (FADN) sample ((More details on the FADN estimation were reported older versions of this section (originally drafted by Markus Kempen and Eoghan Garvey) the CAPRI documentation, accessible in the /doc folder of any stable release of the CAPRI system up to star 2.4 from [[https://www.capri-model.org/dokuwiki/doku.php?id=capri:get-capri.]])) and then these estimation results are combined with total labour requirements within a region (or aggregate national input demand reported in the EAA), using a Highest Posterior Density (HPD) estimation framework.
 ===Labour Input Allocation===
-Input coefficients (family labour and paid labour, both in hours, as well as wage regressions for paid labour) were estimated using standard econometrics from single farm records as found in FADN. While many of results from this process are plausible a number of CAPRI estimates of labour input are inaccurate and untrustworthy, not least when fitted values for labour using the econometric coefficients are compared with total regional labour inputs recoverable from FADN data survey weights. To remedy this, a reconciliation process is undertaken to correct figures for labour input by adjusting the labour input coefficients for both total labour and family labour, handled in file gams\inputs\labour_calc.gms.
+Input coefficients (family labour and paid labour, both in hours, as well as wage regressions for paid labour) were estimated using standard econometrics from single farm records as found in FADN. While many of results from this process are plausible a number of CAPRI estimates of labour input are inaccurate and untrustworthy, not least when fitted values for labour using the econometric coefficients are compared with total regional labour inputs recoverable from FADN data survey weights. To remedy this, a reconciliation process is undertaken to correct figures for labour input by adjusting the labour input coefficients for both total labour and family labour, handled in file gams/inputs/labour_calc.gms.
 The reconciliation process has two components. The first component is to fix on a set of plausible estimates for the labour input coefficients (based on the econometric results) while the second involves a final reconciliation, where further adjustments are made to bring the estimates into line with the FADN values for labour inputs. Implementing these two steps involves the following procedures.
@@ Line 1958: / Line 2470: @@
 |  Region  |  crop or aggregate  |  Econometric estimation  |||  HPD solution including  |||
-|:::| |  regional  |  national- \\ including yield  |  national - \\ without yield  |  regional, \\ national, crop \\ aggregates  |  + expert assumption  |  + regional \\ labour supply  |
+|:::| |  regional  |  national- including yield  |  national - without yield  |  regional, national, crop  aggregates  |  + expert assumption  |  + regional labour supply  |
 |Belgium (BL24)|Soft wheat|	31.49|	31.26|	31.49|	24.99|	32.73|	53.88|
 |:::|Sugar beet	|  76.25|	77.39|	76.25|	62.19|	48.27|	68.36|
@@ Line 1978: / Line 2490: @@
 For typical applications of CAPRI, regional projections of labour use are needed. Such projections have been prepared as well in the CAPSTRAT project, using a cohort analysis to separate 2 components of changes over time: (1) an autonomous component, which comprises structural changes due to demographic factors such as ageing, death, disability and early retirement, and (2) a non-autonomous component, which incorporates all other factors that influence changes in farm structure and has been analysed econometrically.
-The results of this analysis are loaded in the context of CAPRI task “Generate trend projection” in file baseline\labour_ageline.gms, but only to serve as one type of bounds for labour use in the contrained trends for European regions. Other bounds are derived from engineering knowledge (or assumptions) on plausible labur use per activity which is based on the initial estimation of labour allocation by activity.
+The results of this analysis are loaded in the context of CAPRI task “Generate trend projection” in file baseline/labour_ageline.gms, but only to serve as one type of bounds for labour use in the contrained trends for European regions. Other bounds are derived from engineering knowledge (or assumptions) on plausible labur use per activity which is based on the initial estimation of labour allocation by activity.
 =====The global database components =====
@@ Line 1989: / Line 2501: @@
 The task requires input data stemming from an external preparation routine which is not a CAPRI module or sub-module. It is executed only on an intermittend basis depending on the availability of new raw data from FAOSTAT and the requirement for an update of the corresponding input data.
-The resulting output from the external preparation routine are six gdx-files that have to be present in the \dat-folder of the CAPRI working directory: (1) commodityBalances, (2) population, (3) ProductionAndRessources, (4) fao_trade_matrix. Input data files (1) to (3) are required for the country related part (A), the trade matrix (4) is required for consolidation part (B).
+The resulting output from the external preparation routine are six gdx-files that have to be present in the /dat-folder of the CAPRI working directory: (1) commodityBalances, (2) population, (3) ProductionAndRessources, (4) fao_trade_matrix. Input data files (1) to (3) are required for the country related part (A), the trade matrix (4) is required for consolidation part (B).
 ===Consolidation of country level data===
@@ Line 1997: / Line 2509: @@
 The head section of the sub-module comprises (a) initialization of FAOSTAT-related and mapping sets which are used in all futher consolidation sections, (b) loading union sets from the CommodityBalances and ProductionAndRessources data files, (c) introducing the land categories relevant for the land use consolidation, (d) introduction of multiplication factors for the mapping of units between FAOSTAT and CAPRI items, and (e) initialization of parameters. The (c) land categories relevant for the land use consolidation are as follows: \\
-{{::code_p_96.png?600|}} \\
+{{::code_p_96.png?600}}
 The first consolidation section is on “Production and Ressources”. After loading the raw data at the beginning, the FAOSTAT units are mapped to CAPRI units via the “unit_map” set and corresponding multiplication factors as provided under (d) in the head section of the program to harmonise the units. After that the data is checked for completeness and various heuristic rules are applied to fill gaps in the data: \\
-{{:code_p_96_2.png?600|}} \\
+{{:code_p_96_2.png?600}}
-After aggregating data for China and some reporting on missing data the consolidated production data is written to the \fao folder in the restart-directory for usage in the following consolidation steps.
+After aggregating data for China and some reporting on missing data the consolidated production data is written to the /fao folder in the restart-directory for usage in the following consolidation steps.
 The next stept consolidates “Commodity Balances” and introduces the sets for the main balance components and demand positions as well as the mapping between the original FAOSTAT item codes and the commodity balance codes. This is another example that any data consolidation combining different data sets (even when coming from the same agency like FAO) needs to consider different coding systems used in those data sets: \\
-{{:code_p_97.png?600|}} \\
+{{:code_p_97.png?600}}
-In addition to the item code and unit matching and the removal of flags, negative observations are removed (except for stock changes) from the data. Gap filling is based on weighted averages and smoothed interpolation. Total demand is added up from single demand positions if missing and single demand positions are scaled to given total demand in case they do not sum up consistently. Finally, stock changes are adjusted to ensure that market balances are closed. The consolidated commodity balance data is written to the \fao-folder in the restart directory for further usage inside the fao_balance_consolidation.
+In addition to the item code and unit matching and the removal of flags, negative observations are removed (except for stock changes) from the data. Gap filling is based on weighted averages and smoothed interpolation. Total demand is added up from single demand positions if missing and single demand positions are scaled to given total demand in case they do not sum up consistently. Finally, stock changes are adjusted to ensure that market balances are closed. The consolidated commodity balance data is written to the /fao-folder in the restart directory for further usage inside the fao_balance_consolidation.
 The next stept combines production and ressources with the data on commodity balances in order to consolidate the land use data. The consolidation procedure for land use categories is a separate sub-routine included under this section:\\
-{{:code_p_97_2.png?600|}} \\
+{{:code_p_97_2.png?600}}
 The land use consolidation step takes care of the mapping between FAOSTAT and CAPRI land use categories, imposes gap filling routines, introduces auxiliary data from UNFCCC and UNSTATS and ensures that nested land use categories consistently sum up to their totals.
@@ Line 2019: / Line 2531: @@
 The land use consistency is solved as an optimization problem ensuring (a) adding up of single crop areas to land use aggegates and (b) imposes constraints stemming from transition probabilities between different UNFCCC land use categories: \\
-{{:code_p_98.png?600|}}\\
+{{:code_p_98.png?600}}
-Finally, crop area levels are rescaled based on the solution from the optimization problem and yields are recalculated accordingly. The consolidated land use data is written to the \fao-folder in the restart directory.
+Finally, crop area levels are rescaled based on the solution from the optimization problem and yields are recalculated accordingly. The consolidated land use data is written to the /fao-folder in the restart directory.
-The next step consolidates data for the milk sector. The FAOSTAT market balances differ from CAPRI in four aspects that require special adjustment in addition to the mapping and gap filling routines. (1) Farm household production is not included in output from CAPRI COCO module but in the data from FAOSTAT, (2) Liquid whey and (3) liquid skimmed milk are considered in FAOSTAT but not in COCO, (4) Raw milk is not disaggregated into a category for final consumption as required by COCO. At the end of the consolidation section the result is written to \fao-folder in the results-directory. This file is also a major input for the CAPRI GLOBAL module (\fao\fao_milkdata_...gdx).
+The next step consolidates data for the milk sector. The FAOSTAT market balances differ from CAPRI in four aspects that require special adjustment in addition to the mapping and gap filling routines. (1) Farm household production is not included in output from CAPRI COCO module but in the data from FAOSTAT, (2) Liquid whey and (3) liquid skimmed milk are considered in FAOSTAT but not in COCO, (4) Raw milk is not disaggregated into a category for final consumption as required by COCO. At the end of the consolidation section the result is written to /fao-folder in the results-directory. This file is also a major input for the CAPRI GLOBAL module (/fao/fao_milkdata_...gdx).
-Data on population only requires adjustments for Serbia, Montenegro, and China which is taken care of in the following step. The aggregated population time series for Serbia and Montenegro from before 2006 is prolonged to the time after whereas the respective disaggregated time series are back-casted to the period before. Data for China is aggregated. The result is written to the \fao-folder in the results-directory which is a major input for the CAPRI task “Build global database”.
+Data on population only requires adjustments for Serbia, Montenegro, and China which is taken care of in the following step. The aggregated population time series for Serbia and Montenegro from before 2006 is prolonged to the time after whereas the respective disaggregated time series are back-casted to the period before. Data for China is aggregated. The result is written to the /fao-folder in the results-directory which is a major input for the CAPRI task “Build global database”.
 ===Consolidation of the trade flow matrix===
@@ Line 2031: / Line 2543: @@
 The consolidation of trade flows is split up across product specific groups to keep the task feasible in terms of computational complexity. The task is split up among 29 groups in total:\\
-{{:code_p_99.png?600|}}\\
+{{:code_p_99.png?600}}
 The whole procedure for creating a consistent data base as a starting point for the CAPRI task “Build global database” consists of two major tasks that are called the “groupSpecific” and “nongroupSpecific” tasks. The first one is the actual consolidation part that is done for each commodity group separately but executed in parallel. The second one is necessary for exporting the results such that they may be exploited via the GUI or be used as major input for the GLOBAL module. \\
-{{:code_p_99_2.png?600|}}\\
+{{:code_p_99_2.png?600}}
-The group specific task starts 29 separate consolidation processes in parallel where the actual consolidation processes are defined in the separate include file “\fao\do_trade_consolidation_for_one_group.gms”.
+The group specific task starts 29 separate consolidation processes in parallel where the actual consolidation processes are defined in the separate include file “/fao/do_trade_consolidation_for_one_group.gms”.
 The trade consolidation part requires specific FAOSTAT trade data related sets that are loaded at the beginning of the include file. There are 18 different types of output reported in the result array.\\
-{{::code_p_100.png?600|}}\\
+{{::code_p_100.png?600}}
 There are also 25 different statistics reported for the time series that are important intermediate indicators for the trade consolidation process. \\
-{{:code_p_100_2.png?600|}}\\
+{{:code_p_100_2.png?600}}
-The trade consolidation consists of eight steps in sequence that are dependent on each other, i.e. each step produces an intermediate output file that is written to the \fao folder in the restart directory for usage in the follow-up steps.
+The trade consolidation consists of eight steps in sequence that are dependent on each other, i.e. each step produces an intermediate output file that is written to the /fao folder in the restart directory for usage in the follow-up steps.
-The process starts with the (1) SELECT step loading the raw trade data from the file “\dat\fao\FAO_trade_matrix_...gdx” which was produced by an external data preparation routine as described under the head section of this chapter. The raw data are just unloaded without any modifications in smaller files containing only the trade flows for one of the 29 product groups which facilitates subsequent processing. The next step is (2) AGGTRADE taking care of cutting off trade below a threshold of 1.E-5 and assigning a dummy variable for the case that trade was above this threshold.
+The process starts with the (1) SELECT step loading the raw trade data from the file “/dat/fao/FAO_trade_matrix_...gdx” which was produced by an external data preparation routine as described under the head section of this chapter. The raw data are just unloaded without any modifications in smaller files containing only the trade flows for one of the 29 product groups which facilitates subsequent processing. The next step is (2) AGGTRADE taking care of cutting off trade below a threshold of 1.E-5 and assigning a dummy variable for the case that trade was above this threshold.
 The following step (3) UVATRADE filters trade flows computes unit values after some filtering procedures and fills gaps of their national times series based on linear interpolation. Time series of the producer price index are also completed based on averaging over different time horizons, on group averages, and on unit values.
@@ Line 2054: / Line 2566: @@
 In the following step (4) STATRADE a trust indicator is computed that allows to assign a trade flow value in case of conflicting notifications between trade partners. It is based on the sum of absolute differences to partner notifications relative to total notified trade.
-{{:code_p_101.png?600|}} \\
+{{:code_p_101.png?600}} \
 The next step (5) TRDTRADE calculates national linear trend lines for quantities, values, unit values and price indices.
@@ Line 2060: / Line 2572: @@
 Step (6) INITRADE prepares the trade data for the final consolidation procedure by calculating expected means of imports, exports and unit values, and by computing the trust indicator, standard errors and expected standard errors for trade quantity and units, and unit values. The trust indicator is used for adjusting the standard errors in the estimation of trade flows between partners. Higher trust indicators result in lower standard errors and lower standard errors lead to smaller deviations from reported trade, i.e. the outcome from the estimation will deviate less from the reportings for more trustworthy partners, and vice versa.
-{{:code_p_101_2.png?600|}} \\
+{{:code_p_101_2.png?600}}
 The computations are accomplished for each commodity separately.
@@ Line 2068: / Line 2580: @@
 Finally, (8) SHOWTRADE stores the consolidated trade flow quantities in a gdx-file for exploitation and inspection.
-The second “nongroupSpecific” task in the trade consolidation part takes care of exporting the consolidated trade data to the \fao-folder in the results directory. This output is a major input for the CAPRI task “Build global database” (“fao_trade_for_global…gdx”). The trade data is complemented with data on conversion coefficients, on extraction rates, mappings between product equivalent and product codes, and between raw and processed goods, production data on the animal sector, and caseinTrade. The export job is included as a separate program under the nongroupSpecific task.
+The second “nongroupSpecific” task in the trade consolidation part takes care of exporting the consolidated trade data to the /fao-folder in the results directory. This output is a major input for the CAPRI task “Build global database” (“fao_trade_for_global…gdx”). The trade data is complemented with data on conversion coefficients, on extraction rates, mappings between product equivalent and product codes, and between raw and processed goods, production data on the animal sector, and caseinTrade. The export job is included as a separate program under the nongroupSpecific task.
-{{:code_p_102.png?600|}} \\
+{{:code_p_102.png?600}}
 ====Task: Build global database====
-The main program ((A "program" refers in this section to a file with CAPRI code for performing certain task or sub-task and which may in turn include other “code files” or “programs”.)) for this task is capri\gams\global_database.gms which collects a number of included files for separate sub-tasks, some of which being trivial, others more complex.
+The main program ((A "program" refers in this section to a file with CAPRI code for performing certain task or sub-task and which may in turn include other “code files” or “programs”.)) for this task is capri/gams/global_database.gms which collects a number of included files for separate sub-tasks, some of which being trivial, others more complex.
 **Figure 9: Overview on key elements in the consolidation of global data (in global_database.gms)**
-{{:figure_8_1_.png?600|}} \\ Source: own illustration \\
+{{:figure_09.png?600|Source: own illustration}}
 The program starts with including three general programs also present (possibly in task specific form) in other main programms plus the steering file (runglobal.gms) with more precise settings for the current run which may come from the GUI or from a batch file((A batch file is a steering file to execute a CAPRI task with all settings that are usually made in the GUI (say which simulation years) expressed equivalently in a certain language in a text file.)):
-{{:code_p_103.png?600|}} \\
+{{:code_p_103.png?600}}
 After these general settings the programm continues in a rather standard manner with a section collecting various declarations of sets and parameters. Among these are the general sets of CAPRI (sets.gms), and the sets specific to the market model (arm_sets.gms) because the purpose of the task is to compile the data needed for the market model at the global level of CAPRI:
-{{:code_p_104.png?600|}} \\
+{{:code_p_104.png?600}}
 The most important data source for task “Build global database” is FAOstat which involves a fairly long file (FAO_codes_new.gms) with sets and cross-sets to map from FAO regions, items, and products into the CAPRI world (defined by the code system in the annex). This serves to map some key data from FAO compiled in the previous task: population (fao_population.gms), commodity balances combined with production and landuse statistics. Furthermore special balances for dairy products are loaded (all in load_fao_data_new.gms).
-{{:code_p_104_2.png?600|}} \\
+{{:code_p_104_2.png?600}}
 The second most important group of data, both historical as well as projections, for the global market model of CAPRI come from the Aglink-Cosimo model((This model is also used by DG Agri for its own outlook and provides important inputs to the CAPRI baseline.)), including its ex post database.
-{{:code_p_104_3.png?600|}} \\
+{{:code_p104_3.png?600}}
   * The first $include file (load_%aglink%_new.gms((A string like %textname% is a placeholder in GAMS code for some other text to be substituted for %textname% during the program execution. In this example it holds the name for the specific Aglink-Cosimo version that should be loaded.)) ) includes the relevant sets to handle the Aglink data, including the cross-sets to map to CAPRI. In addition it also merges a special data set on fish markets with other original Aglink data.
@@ Line 2103: / Line 2616: @@
 The next three $include files cover additional macroeconomic data from UNstats (load_gdp_unstats_new.gms), include and map long run projections beyond the Aglink horizon from the GLOBIOM((The GLOBIOM model is the second model providing key inputs to the CAPRI baseline. It is mainly developed and operated at IIASA.)) model (create_longrun_info.gms, comment “merge FAO and IMPACT 2050 projections is obsolete), and collect prior values for demand elasticities from the literature (collect_literature_elas.gms, whereas demand elasticties from Aglink-Cosimo are ignored).
-{{:code_p_105.png?600|}} \\
+{{:code_p_105.png?600}}
 It may be seen that “create_longrun_info.gms” is active or not depending on a setting from the GUI or a batch file. Similar to the code processing Aglink information it includes sets and mappings to handle the GLOBIOM information. Another similarity with the Aglink related files is that this code basically needs annual adjustments, because some definitions are changing from year to year and there are two GLOBIOM versions to distinguish, one with a certain EU focus, the other one with a perfectly global orientation. Finally, it may be mentioned that the projections are introduced into the CAPRI world mostly in the form of growth factors.
@@ Line 2109: / Line 2622: @@
 The CAPRI market model is spatial and therefore requires data on bilateral trade flows. These are covered in two include files, the first one dealing with the special case of biofuel trade flows, the second one with the general case. \\
-{{:code_p_106.png?600|}} \\
+{{:code_p_106.png?600}}
 Biofuel trade requires a special treatment again because FAOstat does not cover these. Instead, bilateral trade flows are constructed using total exports and imports from AGLINK and trade data from COMEXT, USDA and FO-Licht. By contrast the data for the trade matrix for other commodities is from FAOstat.
@@ Line 2119: / Line 2632: @@
 The second is a transport cost matrix estimation using the original FAOstat trade matrix (so before gap filling and consolidation) and distance related information from CEPII. Together with price information the transport costs are estimated to provide a link between CIF and FOB prices for bilateral tradeflows.
-{{:code_p_106_2.png?600|}} \\
+{{:code_p_106_2.png?600}}
 The next $include file extends the Aglink-Cosimo projections to 2030, if needed, with a trend estimation involving a number of pragmatic modifications (such as the trend line passing trough the last observation). Then the the growth factors computed previously or the default trends are used to estimate a medium term outlook projections for global market balances, prices or GDP. These projections do however not include any consictency checks on closed market balances or similar properties. This is achieved in the baseline calibration only.
-{{:code_p_107.png?600|}} \\
+{{:code_p_107.png?600}}
 Finally, data on trade policy variables such as applied and scheduled tariffs, tariff rate quotas or bilateral trade agreements are collected from the Agricultural Market Access Database (AMAD, obsolete current version) or from the MacMaps database (%macMap%)((See GAMS Documentation on The GAMS Call and Command Line Parameters (https://www.gams.com/latest/docs/UG_GamsCall.html))==on, but not yet activated under Star2.4).
@@ Line 2129: / Line 2642: @@
 The very last include file is probably also the least important one: FAPRI projections had a more important role several years ago, are not updated anymore and presumable affect less than a dozen numbers (if any at all) in the global database compiled in this task:
-{{:code_p_107_2.png?600|}} \\
+{{:code_p_107_2.png?600}}
 =====Policy data=====
@@ Line 2140: / Line 2653: @@
   - ITC-MacMap and ITC-TradeMap database. MacMap includes ad valorem equivalent tariff rates at the 6-digit level of the Harmonized System (HS6), while TradeMap supplies the necessary trade statistics (quantities and prices) for the aggregation.
-The tariff aggregation results are part of the .gdx output of the global module, and can be found in results\global\tariffs.gdx.
+The tariff aggregation results are part of the .gdx output of the global module, and can be found in results/global/tariffs.gdx.
-Although the tariffs in the tariff databases should already reflect the tariff schedules of the implemented Free Trade Agreements (FTA) on global agricultural markets, CAPRI nevertheless explicitly includes data on a number of FTAs. That FTA-specific policy information enters the CAPRI system in the market model calibration workstep (gams\arm\def_tariff.gms, see Table below for the list of implemented FTAs).
+Although the tariffs in the tariff databases should already reflect the tariff schedules of the implemented Free Trade Agreements (FTA) on global agricultural markets, CAPRI nevertheless explicitly includes data on a number of FTAs. That FTA-specific policy information enters the CAPRI system in the market model calibration workstep (gams/arm/def_tariff.gms, see Table below for the list of implemented FTAs).
 **Table 21: Free Trade Agreements considered in CAPRI**
@@ Line 2150: / Line 2663: @@
 Source: own compillation
-Specific trade policy data on Switzerland enters CAPRI both in the tariff aggregation module (in the global) part and also during market model calibration, and often overwrites tariff data from the above sources. The Switzerland-specific datasets in CAPRI are managed by the team at the Federal Office for Agriculture, and the data are based on national trade statistics: Swiss-Impex database and the databases of the TRIMAG tariff aggregation tool((For more information on TRIMAG please refer to Himics, M., Listorti, G., Tonini, A., 2019. Simulated economic impacts in applied trade modelling: A comparison of tariff aggregation approaches. Economic Modelling. doi:10.1016/j.econmod.2019.08.007)). The relevant model code is collected under the subfolder gams\special_ch.
+Specific trade policy data on Switzerland enters CAPRI both in the tariff aggregation module (in the global) part and also during market model calibration, and often overwrites tariff data from the above sources. The Switzerland-specific datasets in CAPRI are managed by the team at the Federal Office for Agriculture, and the data are based on national trade statistics: Swiss-Impex database and the databases of the TRIMAG tariff aggregation tool((For more information on TRIMAG please refer to Himics, M., Listorti, G., Tonini, A., 2019. Simulated economic impacts in applied trade modelling: A comparison of tariff aggregation approaches. Economic Modelling. doi:10.1016/j.econmod.2019.08.007)). The relevant model code is collected under the subfolder gams/special_ch.
 ===Tariffs and Tariff Rate Quotas===
-Data on trade policy instruments other than tariffs (Tariff Rate Quotas, export subsidies, entry price system and flexible levies) enter CAPRI directly in the market model calibration workstep. Note that the ad valorem equivalent tariff rates in MacMap already include an estimated equivalent tariff rate for TRQs. Nevertheless, the CAPRI market model separates TRQs from fixed tariff rates by using a sigmoid function-representation of the TRQ regime switch mechanism((Tariff rates under TRQ vary between the lower in-quota and the higher out-of-quota rates, depending on the quota fill rates. For more details on the methodological approach please visit section [[Market module for agricultural outputs#Endogenous tariffs under Tariff Rate Quotas, flexible levies and the minimum import price regime for fruits and vegetables of the EU]])).
+Data on trade policy instruments other than tariffs (Tariff Rate Quotas, export subsidies, entry price system and flexible levies) enter CAPRI directly in the market model calibration workstep. Note that the ad valorem equivalent tariff rates in MacMap already include an estimated equivalent tariff rate for TRQs. Nevertheless, the CAPRI market model separates TRQs from fixed tariff rates by using a sigmoid function-representation of the TRQ regime switch mechanism((Tariff rates under TRQ vary between the lower in-quota and the higher out-of-quota rates, depending on the quota fill rates. For more details on the methodological approach please visit section [[scenario simulation#Endogenous tariffs under Tariff Rate Quotas, flexible levies and the minimum import price regime for fruits and vegetables of the EU]])).
-The TRQ system of the EU is included in great detail, based on DG AGRI.information. Data on  TRQ orders are aggregated to the geographical and commodity definitions of CAPRI in dat\arm\TRQ_orderds.gms. Specific GAMS routines convert some of the compound TRQs into ad valorem TRQs if necessary((Compound TRQs are TRQs applying a compound tariff (combination of specific and ad valorem) on the in-quota or out-of-quota imports. For methodological reasons, the compound tariffs might need to be converted into their ad valorem equivalent rates.))(gams\arm\convert_compound_trqs.gms).
+The TRQ system of the EU is included in great detail, based on DG AGRI.information. Data on  TRQ orders are aggregated to the geographical and commodity definitions of CAPRI in dat/arm/TRQ_orderds.gms. Specific GAMS routines convert some of the compound TRQs into ad valorem TRQs if necessary((Compound TRQs are TRQs applying a compound tariff (combination of specific and ad valorem) on the in-quota or out-of-quota imports. For methodological reasons, the compound tariffs might need to be converted into their ad valorem equivalent rates.))(gams/arm/convert_compound_trqs.gms).
 ===Export subsidies===
-Data on (EU) export subsidies (e.g. maximum commitments) enter the system in the market model calibration workstep, under gams\arm\calc_feoga.gms. Current WTO negotiations aim at the full phase-out of export subsidies, and accordingly, the EU does not grant export subsidies to agricultural products currently. Nevertheless, the possibility to introduce export subsidies in policy scenarios is kept in CAPRI (e.g. Border Carbon Adjustment policies may take the form of export subsidies, for which the availability of the export subsidy mechanism is valuable).
+Data on (EU) export subsidies (e.g. maximum commitments) enter the system in the market model calibration workstep, under gams/arm/calc_feoga.gms. Current WTO negotiations aim at the full phase-out of export subsidies, and accordingly, the EU does not grant export subsidies to agricultural products currently. Nevertheless, the possibility to introduce export subsidies in policy scenarios is kept in CAPRI (e.g. Border Carbon Adjustment policies may take the form of export subsidies, for which the availability of the export subsidy mechanism is valuable).
 ===Producer subsidies===
@@ Line 2171: / Line 2684: @@
 ===Public Intervention purchases and sales ===
-Data on public intervention (stocks, buy-ins, releases, administrative prices etc.) enter the system in the market model calibration workstep, under gams\arm\calc_feoga.gms. Once one of the most impactful measure of the Common Agricultural Policy (CAP), public intervention has been reduced regarding its scope and is currently only available for EU farmers as an emergency measure (in crisis situations, e.g. under exceptionally high price fluctuations). Therefore, its use in CAPRI is also limited to scenario analysis.
+Data on public intervention (stocks, buy-ins, releases, administrative prices etc.) enter the system in the market model calibration workstep, under gams/arm/calc_feoga.gms. Once one of the most impactful measure of the Common Agricultural Policy (CAP), public intervention has been reduced regarding its scope and is currently only available for EU farmers as an emergency measure (in crisis situations, e.g. under exceptionally high price fluctuations). Therefore, its use in CAPRI is also limited to scenario analysis.
  **//Further update of this section is pending//**