Differences

This shows you the differences between two versions of the page.

--- module_for_agricultural_supply_at_regional_level [2020/03/11 10:31] – [Detailed discussion of the equations in the supply model] matsz
+++ module_for_agricultural_supply_at_regional_level [2022/11/07 10:23] – external edit 127.0.0.1
@@ Line 89: / Line 89: @@
 Where “asym” is the land asymptote, i.e. the maximal amount of economically usable agricultural area in a region when the agricultural land rent goes towards infinity. For an application where the land market is used see Renwick et al. (2013).
-Set aside policies have changed frequently during CAP reforms. The recent specification is covered in the context of the premium modelling in Section 5.3 FIXME . The obligatory set-aside restriction introduced by the McSharry reform 1992 and valid until the implementation of the Luxembourg compromise of June 2003 has been explicitly modelled through this equation:
+Set aside policies have changed frequently during CAP reforms. The recent specification is covered in the context of the premium modelling in Section [[Premium module]]. The obligatory set-aside restriction introduced by the McSharry reform 1992 and valid until the implementation of the Luxembourg compromise of June 2003 has been explicitly modelled through this equation:
 \begin{align}
@@ Line 125: / Line 125: @@
 \begin{align}
 \begin{split}
-&LEVL_{r,"iset"} + LEVL_{r,"gset"} + LEVL_{r,"tset"}+ LEVL_{r,"vset"} \\
+&\sum_{i \in I \, j,k} \left[ levl_{rik} \left ( ret_{rni} (1-biofix_{rni}) \lambda_{rnik}^{prop} + \lambda_{rni}^{const} \right ) soil_{rn} \, yf_{rnik} \right ] \\
-& \le \sum_{arab \wedge SETF_{r,arab}} LEVL_{r,arab}/\overline{MXSETA}
+& = fmine_{rni}(1-loss_{rn}) +fexcr_{rni} \phi_{r,excr,n} +(1-isPerm_j)fcres_{rni}(1-loss_{rn})\phi_{r,cres,n}\\
+& + isPerm_j  \sum_{ i \in I \, j,k} levl_{rik}res_{rni}(techf_{rink}+1)(1-loss_{rn})\phi_r,cres,n \\
+& \forall r,n,j
+\end{split}
+\end{align} FIXME
+Indices:
+\(r\) = region \\
+\(i\) = crop \\
+\(j\) = crop group \\
+\(k\) = technological crop option (high/low yield) \\
+\(n\) = nutrient (N/P/K) \\
+\(isPerm_j\) = indicates that crop group \(j\) contains permanent crops
+Endogenous choice variables:
+\(levl_{rik}\) = Area (ha) of each crop \(i\) and technology \(k\) in region \(r\).  \\
+\(fmine_{rnj}\) = Application of mineral fertilizer \(n\) to crop group \(j\) in region \(r\).  \\
+\(fexcr_{rnj}\) = Application of manure \(n\) to crop group \(j\) in region \(r\).  \\
+\(fcrex_{rnj}\) = Allocation of crop residue \(n\) to crop group \(j\) in region \(r\).
+Parameters:
+\(ret_{rni}\) 	=	Retention (uptake) of nutrients by the crop	\\
+\(res_{rni}\) 	=	Crop residues output \\
+\(biofix_{rni}\)  =	Biological fixation, share (only for N and selected crops) \\
+\(λ_{rnik}^{prop} \)	=	Over-fertilization factor, calibrated \\
+\(λ_{rni}^{const} \)	=	Over-fertilization term, calibrated \\
+\(soil_{rn}\) 	=	Soil factor \\
+\(yf_{rnik}\) 	=	Yield factor for technologies \\
+\(loss_{rn}\)	=	Loss rate \\
+\(ϕ_{r",excr" ,n}\) = 	Nutrient availability ratio for manure \\
+\(ϕ_{r,"cres" ,n}\) =	Nutrient availability ratio for crop residues \\
+The reader may have noted that there is no loss rate for manure in the Equation 96 FIXME . CAPRI does contain such loss rates, but they are specific for each animal type and therefore happens on the manure supply side of the regional manure balance (see section on input allocation).
+The model contains three types of manure: N-manure, P-manure and K-manure. From an agricultural point of view this may seem odd. It might be more intuitive to think of one type of manure per animal category. The motivation is to keep the system simple and flexible. With the present representation, where each animal category supplies N, P, and K-manure, the number of manure classes can be limited and yet the unique mix of nutrients from each animal category can be defined.
+The supply of each manure type is collected in a “pool” for each regional farm model, i.e. for each NUTS2 region. Regions within a member state may trade manure, subject to a cost. The supply in the pool plus the traded quantities has to be distributed to the crops in the region, i.e. there is an equality-restriction in place. This is handled in the equations “FertDistExcr_” and “ManureNPK_”. Note that fertilizer flows are measured in //tons//, for the sake of scaling, whereas other total quantities in CAPRI are measured in 1000 tons. Hence the factors 1000 and 0.001.
+\begin{equation}
+\sum_j fexcr_{rnj}= 1000 v\_ManureNPK_{rn}
+\end{equation}
+\begin{equation}
+v\_ManureNPK_{rn} + \sum_s T_{rs}nutshr_{rn} = 0.001 \sum_{i\in Anim_j,k} levl_{rik}o_{rnik}(1-loss_{rin}) \quad \forall r, n
+\end{equation}
+where \\
+\(o_{rnik}\) 	is the output of manure nutrient \(n\) from animal type \(i\) using technology \(k\) in region \(r\), \\
+\(nutshr_{rn}\)	is the average content of each nutrient in the regional manure pool, \\
+\(T_{rs}\)	is the quantity of manure traded from \(r\) to \(s\), \\
+\(isAnim_{i}\)	indicates that activity \(i\) is an animal production activity
+Equation “FertDistMine_” allocates total mineral fertilizer sales to the crops / group of crops.
+\begin{equation}
+\sum_j fmine_{rnj} = -netPutQuant_{rn}
+\end{equation}
+Finally, crop residues and atmospheric definition are distributed in equation “FertDistCres_”.
+\begin{equation}
+\sum_j fcers_{rnj} = \sum_{i \notin isPerm_j,k} levl_{rik}res_{rni}(techf_{rink}+1)
+\end{equation}
+One flow from a source s={"mine" ,"cres" ,"excr" } to a sink j={"crop groups"} can in general be anything from zero and upwards. The nutrient balance equations above do not uniquely determine each flow of nutrients from sources to sinks, but it is indeed possible that in one simulation, say, a particular crop group gets much crop residues and little manure, whereas the opposite holds in the next simulation. The total balances will hold equally well in either situation, and the profits will not be affected since the same total amount of mineral fertilizer is purchased, but we do have a stability problem for the model. Furthermore, the different nutrient flows may influence the greenhouse gas emission coefficients of crops (if e.g. the emissions of enteric fermentation follows the manure to the crops). The problem is under-determined, or ill-posed.
+To resolve the ill-posedness of the fertilizer distribution, we propose a probabilistic approach. This means that we do not introduce any additional economic model for the allocation that somehow makes increasing fertilizer flows more expensive. Instead, we assume that whatever the reasons the farmers have for choosing a particular distribution, those reasons are similar in two simulations, and therefore the fertilizer flows are also similar. Thus, a larger deviation from some reference flows is deemed improbable, albeit not costlier than the situation with the reference flows.
+To develop this probabilistic model, we assume that the decisions of the farmer are separable and taken in two steps: first, the farmer decides about the cropping plan and just ensures that the total amount of fertilizer available is sufficient. This is called the outer model. Then, a statistical model is solved that finds the most probable fertilizer flows out of the continuum of possible ones. This is called the inner model. The structure with outer and inner models makes the problem a bi-level programming one.
+To implement the bi-level programming problem in a way that does not change the present structure of the model (with just one optimization solve of the representative farm model) we implement the inner model by its optimality conditions. By carefully choosing the proper probability density functions we ensure that no complementary slackness conditions are needed, so that the inner model is simple to solve. For this the gamma density function is very suitable, as it has a support from zero to infinity, with a probability that goes towards zero as the random variable goes to zero.
+The parameters of the gamma function are determined in the calibration step, described further below, and then kept constant in simulation. The gamma density function for some random variable x has the form
+\begin{equation}
+p(x|\alpha,\beta) = \frac {\beta^{\alpha}}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x}
+\end{equation}
+where Γ(α) is the gamma function, and α and β are parameters that determine the shape of the density function. The gamma density is nonlinear, and the joint density, being the product of the densities of all nutrient flows, is even more so. In order to reduce nonlinearity we note that are interested in finding the highest posterior density, i.e. maximizing a joint density function, and since the maximum is invariant to monotonous positive transformations we compute the logarithm of the joint density, which will be the sum of terms like the following (the constant term has been omitted since it also does not influence the optimal solution for x):
+\begin{equation}
+log \, p(x|\alpha,\beta) \propto (\alpha-1)log \, x-\beta x
+\end{equation}
+Maximization of the logged density under the constraints that the nutrient balance restrictions of the supply modes have to be met gives a set of equations that define explicit and unique fertilizer flows, where v are the Lagrange multipliers of the source-pool restrictions and u the Lagrange multipliers of the nutrient balance equation.
+\begin{equation}
+\text{FOC w.r.t. manure use:} \\
+\frac {\alpha_{r,excr,nj}-1}{fexcr_{rnj}} - \beta_{r,excr,nj} - v_{r,excr,n}+\phi_{r,excr,n}u_{rnj} = 0
+\end{equation}
+\begin{equation}
+\text{FOC w.r.t. crop residues use:} \\
+\frac {\alpha_{r,cres,nj}-1}{fcres_{rnj}} - \beta_{r,cres,nj} - v_{r,cres,n}+(1-loss_{rn}\phi_{r,excr,n}u_{rnj} = 0 \forall j\notin isPerm_j
+\end{equation}
+\begin{equation}
+\text{FOC w.r.t. mineral feritilzer use:} \\
+\frac {\alpha_{r,mine,nj}-1}{fmine_{rnj}} - \beta_{r,mine,nj} - v_{r,mine,n}+(1-loss_{rn}u_{rnj} = 0
+\end{equation}
+The system of FOC contains expressions of the type \(1/fmine\) which is likely to impair performance as the second derivatives are not constant (CONOPT computes second derivatives). Therefore, the first term in each FOC was turned into a new variable \(z\) defined as \( z_{r,"excr" ,nj}fexcr_{r,"excr" ,nj}=\alpha_{r,"excr" ,nj}-1\), and similar for each source, which is a quadratic expression.
+===Balancing equations for outputs===
+Outputs produced must be sold – if they are tradable across regions – or used internally, as in the case of young animals or feed.
+\begin{equation}
+\sum_{act}Levl_{r,act}OUTP_{r,act,o}=NETTRD_r^{o\notin fodder}+YANUSE_r^{o\notin oyani} +FEDUSE_r^{o\in fodder}
+\end{equation}
+In the case of quotas (milk, for sugar beet) the sales to the market may be bounded (noting that NETTRD = v_netPutQuant in the code):
+{{::code_p_150.png?600|}}
+As described in the data base chapter, the concept of the EAA requires a distinction between young animals as inputs and outputs, where only the net trade is valued in the EAA on the output side. Consequently, the remonte expressed as demand for young animals on the input side must be mapped into equivalent ‘net import’ of young animals on the output side:
+\begin{equation}
+\sum_{aact}Levl_{r,aact}I_{r,aact,yani}=YANUSE_r^{oyani \leftrightarrow iyani}
+\end{equation}
+In combination with the standard balancing equation shown above, the NETTRD variable for young animals on the output side becomes negative if the YANUSE variable for a certain type of young animals exceeds the production inside the region.
+===The objective function===
+The objective function is split up into the linear part, the one related to the quadratic cost function for activities, and the quadratic cost function related to the feed mix costs:
+\begin{equation}
+OBJE=\sum_r LINEAR_r+QUADRA_r+QUADRF_r
+\end{equation}
+The linear part comprises the revenues from sales and the costs of purchases, minus the costs of allocated inputs not explicitly covered by constraints (i.e. all inputs with the exemptions of fertilisers, feed and young animals) plus premiums:
+\begin{equation}
+LINEAR_r= \sum_{io} NETTRD_{r,io} \overline{ PRICE}_{io}+ \sum_act LEVL_{r,act}\left (\overline{PRME}_{r,act}-\overline{COST}_{r,act}\right)
+\end{equation}
+The quadratic cost function relating to feed is defined as follows:
+\begin{equation}
+QUDRAF_r = \sum_{aact,feed}  \left[
+\begin {matrix}
+LEVL_{r,aact}FEDNG_{r,aact,feed} \\
+(a_{r,aact,feed}+ 1/2b_{r,aact,feed}FEDNG_{r,aact,feed})
+\end{matrix}
+\right]
+\end{equation}
+The marginal feed costs per animal increase hence linearly with an increase in the feed input coefficients per animal. It should be emphasised that this is the main mechanism that “stabilises” the feed allocation by animals. The two balances on feed energy and protein alone would otherwise leave the feed allocation indeterminate and give a rather “jumpy” simulation behaviour.
+There is another more complex PMP term (equation quadra_ in supply_model.gms, not reproduced in this section) quadratic in activity levels and differentiated by the two technologies that “stabilises” the composition of activites according to previous econometric estimates or default assumptions.
+A final term relates to the entitlements introduced with the 2003 Mid Term reviews. If those entitlements are overshot, a penalty term equal to the premium paid under the respective scheme (regional, historical etc.) is subtracted to the objective. Accordingly, the marginal premium for an additional ha above the entitlement ceiling is zero.
+===Sugar beet (M. Adenäuer, P. Witzke)===
+The Common Market Organisation (CMO) for sugar regulates European sugar beet supply with a system of production quotas, even after the significant reforms of 2006, up to year 2017 when the quota system expired. Before that reform, two different quotas had been established subject to different price guarantee (A and B quotas, qA and qB). Beet prices were depending on intervention prices and levies to finance the subsidised export of a part of the quota production to third countries. Sugar beets produced beyond those quotas (so called C beets) were sold as sugar on the world market at prevailing prices, i.e. formally without subsidies. However, a WTO panel initiated by Australia and Brazil concluded that the former sugar CMO involved a cross-subsidisation of C-sugar from quota sugar such that all exports of C sugar was also counted in terms of the EU’s limits on subsidised exports. As a consequence, this outlet for EU surplus production was closed. The reformed CMO therefore does not allow any exports beyond the Uruguay round limits. Instead, processing of beets to ethanol emerged as a new outlet that economically plays a similar role as former C beet production: It offers an outlet for high production quantities that exceed the quota limits of farmers, but at a reduced price. Basically, farmers face a kinked beet demand curve that potentially involved three price levels:
+  * A-beets receiving the highest price derived from high sugar prices (and before the 2006 reform less a small levy amount)
+  * B-beets receiving a lower price as the applicable levies were higher before the reform. However, the 2006 reform eliminated the distinction of A and B quotas. Furthermore, the sugar industry applied a pooling price system in many MS that also eliminated the distinction between A and B beets.
+  * C-beets receiving the lowest price, formerly derived from world market sugar prices, now derived from ethanol prices.
+The high price sector covers for farmers at least the farm level quota endowment. However, the sugar industry may grant high prices also for a limited, “desirable” over-quota production, for example to avoid bottlenecks in sugar or ethanol production. This has been the case in some EU countries before the reform (so-called “C1 beets”) and it is also current practice (see, for example [[http://www.liz-online.de]]).
+Considering a kinked demand curve and in addition yield uncertainty renders the standard profit maximisation hypothesis inappropriate for the sugar sector (at least). The CAPRI system therefore applies an expected profit maximisation framework that takes care for yield uncertainty (see Adenäuer 2005). The idea behind this is that observed C sugar productions in the past are unlikely to be an outcome of competitiveness at C beet prices rather than being the result of farmers’ aspirations to fulfil their quota rights even in case of a bad harvest. This approach essentially assumes that the “behavioural quotas” of farmers may exceed the “legal quotas” (derived from the sugar CMO) by some percentage. This percentage reflects in part the pricing behaviour of the regional sugar industry, but it may also depend on farmers expectations on the consequences of an incomplete quota fill. These aspects may be captured with the following specification of expected sugar beet revenues that substitute for the expression \(NETTRD_{r,io} \; PRICE_{io} \) (if \(io=SUGB\)) in equation below:
+\begin{align}
+\begin{split}
+SegbREV_r & = p^A NETTRD_{r,SUGB} \\
+& - \left( p^A-p^B\right) \left [
+\begin{matrix}
+(1-CDFSugb(q^A))(NETTRD_{r,SUGB}-q^A) \\
++ (\sigma^S)^2 PDFSugb(q^A)
+\end{matrix}
+\right] \\
+& -\left(p^B-p^C\right) \left [
+\begin{matrix}
+(1-CDFSugb(q^A+B))(NETTRD_{r,SUGB}-q^A+B) \\
++ (\sigma^S)^2 PDFSugb(q^A+B)
+\end{matrix}
+\right]
 \end{split}
 \end{align}
+Where \(PDFSugb_r\) and \(CDFSugb_r\) are the probability res. cumulated density functions of the NETTRD variable with the standard deviation \(\sigma^S\). \(\sigma^S\) is defined as \(NETTRD_{r,SUGB} * VCOF_r\), where the latter is the regional coefficient of yield variation estimated from FADN. \({p^ABC}\) are the prices for the three different types of sugar beet which are exogenous and linked to the EU and world market prices for sugar. The quotas \(q^A\) and \(q^{A+B}\)  used in Equation 111 FIXME are the “behavioural quotas, currently specified as follows:
+\begin{align}
+\begin{split}
+p^A &= legalqout^A \cdot scalefac \\
+& = legalqout^A \cdot \left(\frac{NETTRD_{SUGB}^{cal}}{legalqout^A} \right )^{0.8}
+\end{split}
+\end{align}
+The scaling factor to map from the legal quota legalquotA (as the B quota has been eliminated in the sugar reform, it holds that \(q^A = q^{A+B}) \)to the behavioural quota qA depends on the projected sugar beet sales quantity in the calibration point \(NETTRD_{SUGB}^{cal} : For a country with a high over quota production (say 40%) we would obtain a scaling factor of 1.31, such that this producer will behave like a moderate C-sugar producer: responsive to both the C-beet prices as well as to the quota beet price (and the legal quotas). Without this scaling factor, producers with significant over quota p   roduction, like France and Germany, would not show any sizeable response to a 10% cut of either the legal quotas or the quota price (at empirically observed coefficients of variation). As it is likely that the profitability of ethanol beets benefit from cross-subsidisation from the quota beets such a zero responsiveness was considered implausible.
+===Update note===
+A number of recent developments are not covered in the previous exposition of supply model equations
+  -A series of projects have added a distinction of rainfed and irrigated varieties of most crop activities which is the core of the so-called “CAPRI-water” version of the system((A more complete presentation is given in [[https://ec.europa.eu/jrc/en/publication/eur-scientific-and-technical-research-reports/extension-capri-model-irrigation-sub-module]].)).
+  -Several projects have added endogenous GHG mitigation options((These are most completely included in the “trunk” version of the CAPRI system. For details, see, for example, [[http://publications.jrc.ec.europa.eu/repository/bitstream/JRC101396/jrc101396_ecampa2_final_report.pdf]].))
+  -Several new equations serve to explicitly represent environmental constraints deriving from the Nitrates Directive and the NEC directive((These are most completely included in the “trunk” version of the CAPRI system but developments are still ongoing.)).
+  -A complete area balance monitoring the land use changes according to the six UNFCCC land use types (cropland, grassland, forest land, wetland, settlements, residual land) has been introduced for carbon accounting
+====Calibration of the regional programming models====
+Since the very first CAPRI version, ideas based on Positive Mathematical Programming were used to achieve perfect calibration to observed behaviour – namely regional statistics on cropping pattern, herds and yield – and data base results as the input or feed distribution. The basic idea is to interpret the ‘observed’ situation as a profit maximising choice of the agent, assuming that all constraints and coefficients are correctly specified with the exemption of costs or revenues not included in the model. Any difference between the marginal revenues and the marginal costs found at the base year situation is then mapped into a non-linear cost function, so that marginal revenues and costs are equal for all activities. In order to find the difference between marginal costs and revenues in the model without the non-linear cost function, calibration bounds around the choice variables are introduced.
+The reader is now reminded that marginal costs in a programming model without non-linear terms comprise the accounting cost found in the objective and opportunity costs linked to binding resources. The opportunity costs in turn are a function of the accounting costs found in the objective. It is therefore not astonishing that a model where marginal revenues are not equal to marginal revenues at observed activity levels will most probably not produce reliable estimates of opportunity costs. The CAPRI team responded to that problem by defining exogenously the opportunity costs of two major restrictions: for the land balance and for milk quotas. The remaining shadow prices mostly relate to the feed block, and are less critical as they have a clear connection to prices of marketable feed as cereals which are not subject to the problems discussed above.
+====Estimating the supply response of the regional programming models====
+The development, test and validation of econometric approaches to estimate supply responses at the regional level in the context of regional programming models form an important task for the CAPRI team. Up to now, there is still no fully satisfactory solution of the problem, but some of the approaches are discussed in here.
+The two possible competitors are standard duality based approaches with a following calibration step or estimates based directly on the Kuhn-Tucker conditions of the programming models. Both may or may not require a priori information to overcome missing degrees of freedom or reduce second or higher moments of estimated parameters. The duality based system estimation approach has the advantage to be well established. Less data is required for the estimation, typically prices and premiums and production quantities. That may be seen as advantage to reduce the amount of more or less constructed information entering the estimation, as input coefficients. However, the calibration process is cumbersome, and the resulting elasticities in simulation experiments will differ from the results of the econometric analysis.
+The second approach – estimating parameters using the Kuhn-Tucker-conditions of the model – leads clearly to consistency between the estimation and simulation framework. However, for a model with as many choice variables as CAPRI that straightforward approach may require modifications as well, e.g. by defining the opportunity costs from the feed requirements exogenously.
+The dissertation work of Torbjoern Jansson (Jansson 2007) focussed on estimating the CAPRI supply side parameters. The results have been incorporated in the current version. The milk study (2007/08) contributed additional empirical evidence on marginal costs related to milk production (Kempen et al. 2011)
+====Price depending crop yields and input coefficients ====
+Let Y denote yields and j production activities. Yield react via iso-elastic functions to changes in output prices
+\begin{equation}
+log(Y_j)=\alpha_j+\epsilon_j \, log(p_o)
+\end{equation}
+The current implementation features yield elasticities for cereals chosen as 0.3, and for oilseeds and potatoes chosen as 0.2. These estimates might be somewhat conservative when compared e.g. with Keeney and Hertel (2008). However, in CAPRI they relate to small scale regional units and single crops, and to European conditions which might be characterized by a combination of higher incentive for extensive management practises and dominance of rainfed agriculture where water might be a yield limiting factor.
+Currently, the code is set up as to only capture the effect of output prices. However, in order to spare calculation of the constant terms α, the actual code implemented in ‘//endog_yields.gms//’ change the yields iteratively in between iterations //t//, using relative changes:
+\begin{equation}
+Y_{j,t}=Y_{j,t-1}^{[\epsilon_jlog \frac{p_o,t-1}{p_t}]}
+\end{equation}