Historical forecasts

 

Below you find actual forecasts from BASMOD put forward by the Riksbank (RB) between 2000:1 and 2003:2. 2003 is the last year for which actual values are observable.

 

Table 1. Evaluation of forecasts in BASMOD. Historical forecasts. Actuals within parentheses. Forecasts with VAR within []. Forecasts with random walk within {}. Riksbank (RB) forecasts within . Forecasts in Rixmod within . Forecasts by NIER within .

 

Inflation CPI

GDP-growth

Start of fore-cast

Current year

About 1 year ahead

About 2 years ahead

Current year

About 1 year ahead

About 2 years ahead

2000:1

1,2 (1.0)

[1.9] {0,8}

1,9 (2,4)

[1.5] {0,4}

2,5 (2,2)

[1.5]{0,0}

4,1 (4,4)

[5.3] {5,0}

 

4,0 (1,2)

[4.1]{5,0}

 

3,2 (2,0)

[3.9]{5,1}

 

2000:3

1,0 (1,0)

[1.3] {0,9}

 

1,3 (2,4)

[2.4] {0,5}

 

1,6 (2,2)

[2.0]{0,1}

4,2 (4,4)

[4.9] {5,2}

 

4,5 (1,2)

[4.3]{5,4}

 

3,5 (2,0)

[4.0]{5,5}

 

2001:1

2,6 (2,4)

[2.0] {1,0}

 

2,4 (2,2)

[1.5] {0,6}

 

1,9 (2,0)

[1.4]{0,2}

 

1,9 (1,2)

[2.7] {3,2}

 

3,8 (2,0)

[3.3]{3,2}

 

2,2 (1,6)

[3.5]{3,2}

 

2001:2

3 (2,4)

[1.4] {1,5}

3,5 (2,2)

[1.3] {1,2}

2,3 (2,0)

[1.2]{0,8}

1,5 (1,2)

[2.0] {2,3}

1,6 (2,0)

[3.2]{2,2}

2,6 (1,6)

[3.4]{2,2}

2002:3

-

-

-

2,0 (2,0)

[2.1] {2,2}

1,8 (1,6)

[2.5]{2,4}

2,5

 

2003:1

2,4 (2,0)

[1,4] {1,9}

2,0

 

 

2,0

 

 

1,8 (1,6)

[1,2] {1,7}

2,4

 

 

2,7

 

 

2003:2

2,5 (2,0)

[2,6] {3,0}

1,9

2,0

1,5 (1,6)

[1,3] {1,6}

3

2,9

RMSE

 

0,38* (0)

[0,75]{0,69}

 *

0,89 (0)

[0,73]{1,67}

 *

0,37* (0)

[0,62]{1,87}

0,33* (0)

[0,60]{0,95}

2,10 (0)

[2,10]{2,62}

 *

1,12 (0)

[1,90]{2,49}

 *

 

Forecasts are compared with VAR, random walk, Riksbank (Monetary Policy Department plus eventual judgements from the Board of Governors) and Rixmod which is another big macroeconomic model used at the department. Forecasts were done for the preceding year and for about 1 and 2 years ahead. All in all there are 16 forecasts to evaluate. The first thing to note is that the random walk is clearly beaten by all the competitors (except for the current year inflation forecast where the VAR model is beaten by the random walk and for the current year GDP growth rate where Rixmod is beaten). Another thing to note is that both the big models perform well and generally beats the VAR model. The exception here is that the current year forecasts by Rixmod are poor. But Rixmod on the other hand has the best GDP growth forecasts for 2 and 3 years ahead. One can also note that BASMOD produces the best current year forecasts for both inflation and GDP growth.

 

I also computed the RMSE for the inflation forecast 2 percent (ó the inflation target). It turned out to be 0.4 which means that it on average beat all the models here. It doesn’t beat BASMOD on the current year forecast and is similar to the Riksbank second year forecast but beats all models for the third year forecast.

 

Table 2 shows a ranking of the forecasting performance on different horizons. BASMOD wins 3, Rixmod 2 and the Riksbank 1 of the competitions. The VAR model comes second for current year GDP growth forecast at best, but the winner BASMOD then has only 55 percent of the errors of the VAR model.

 

Table 2. Ranking of forecast performance on different horizons. Figures within parentheses show the winner’s errors relative to the competitor’s errors in percent.

Ranking of the forecasting performance for inflation.

 

Current year

Next year

Next next year

1

BASMOD/NIER

Riksbank

BASMOD

2

Riksbank (72)

NIER (73)

Rixmod (63)

3

Rixmod (60)

Rixmod (64)

VAR (60)

4

RW (55)

VAR (63)

RW (20)

5

VAR (51)

BASMOD (52)

 

6

 

RW (28)

 

Ranking of the forecasting performance for GDP growth.

 

Current year

Next year

Next next year

1

BASMOD

Rixmod

Rixmod

2

VAR (55)

Riksbank (75)

Riksbank (57)

3

Riksbank (50)

NIER (70)

BASMOD (53)

4

NIER (45)

BASMOD/VAR (56)

VAR (31)

5

RW (35)

RW (45)

RW (24)

6

Rixmod (22)

 

 

 

Although the number of forecasts here is small and one should be careful to draw firm conclusions, the following results emerged:

 

·        the Riksbank performs well on inflation forecasts 1-2 years ahead

·        additional judgements by the Riksbank to the big models BASMOD and Rixmod add very little improvements to the forecasts

·        both the big models perform well

·        BASMOD performs best in the short run (1 year ahead or less)

·        all forecasts beat the random walk

 

 

Table 3. Summary of RMSE in historical forecasts (mean for all horizons).

 

BASMOD

VAR

RW

Riksbank

Rixmod

CPI-inflation

0.56

0.68

1.44

0.50

0.64

GDP-growth

1.32

1.59

2.02

1.12

1.21

 

 

Simulated forecasts

 

The evaluations above were done for the actual forecasts where the number of forecasts is small. The advantage with that type of evaluations is that you can be sure that no more information than is actually available has been used. The forecasts for the VAR and the random walk model were however produced ex post and should not have been a disadvantage for them, rather the opposite since revised and not preliminary data were used for those models. To make a comparison under similar circumstances I now generate forecasts with BASMOD and the VAR.

 

I forecast GDP growth and inflation in BASMOD and the competing VAR model used above. Both models are estimated up to approximately 2003:4 or at the most until 2004:2. In BASMOD some equations are not updated and the sample period varies in different equations. In the VAR model I use data up to 2004:2. The estimated VAR model is supposed to be close to one of the models used at the Monetary Policy Department and which has proved to produce reasonable forecasts. The VAR model uses the following variables

 

·        inflation

·        GDP

·        rate of unemployment

·        unit labour cost

·        short run interest rate

·        effective exchange rate

·        world GDP

·        world inflation

·        world interest rate

 

 

and is estimated in first differences except for interest rates, unemployment and effective exchange rate which are estimated in levels. As in the departments’ model, 4 lags are used. Forecasts are run three years ahead starting 1997-1999 and ending 2001-2003. Forecasts are evaluated at an annual basis to comply with the procedures in the Inflation Report.

 

Table 4 shows the forecasts for GDP growth. On average BASMOD produces slightly more accurate forecasts than the VAR model. This is the case for all horizons. BASMOD gives larger errors in the forecasts starting in 1997 and 1998 but lower errors for 1999-2001.On average for all horizons and periods the forecast errors are about 25 percent lower in BASMOD than in the VAR model.

 

 

Table 4. Forecast of annual GDP growth rates. Forecast errors and RMSE.

 

1997

1998

1999

2000

2001

RMSE

VAR

0.19

-0.35

0.87

0.64

-1.32

0.78

BASMOD

-0.30

0.22

0.78

0.58

0.55

0.53

 

 

 

 

 

 

 

 

1998

1999

2000

2001

2002

 

VAR

-0.59

0.75

1.31

-1.22

-0.65

0.95

BASMOD

0.71

1.60

0.81

-0.24

0.30

0.88

 

 

 

 

 

 

 

 

1999

2000

2001

2002

2003

 

VAR

0.18

1.29

-1.19

-0.13

1.95

1.18

BASMOD

1.85

0.71

-0.42

-0.20

0.22

0.92

 

RMSE

RMSE

RMSE

RMSE

RMSE

 

VAR

0.37

0.88

1.14

0.80

0.88

0.98

BASMOD

1.16

1.02

0.69

0.38

0.38

0.75

 

 

Table 5. Forecast of annual inflation rates. Forecast errors and RMSE.

 

1997

1998

1999

2000

2001

RMSE

VAR

0.55

-0.75

0.34

-0.99

-0.26

0.64

BASMOD

-0.29

-1.07

-0.36

0.0

0.9

0.66

 

 

 

 

 

 

 

 

1998

1999

2000

2001

2002

 

VAR

-0.74

-0.69

-0.15

-0.59

-0.77

0.63

BASMOD

-1.01

-0.92

-0.03

1.1

1.0

0.90

 

 

 

 

 

 

 

 

1999

2000

2001

2002

2003

 

VAR

-0.81

-0.68

0.27

-0.49

-1.01

0.70

BASMOD

-0.94

-0.06

1.06

0.9

0.4

0.77

 

RMSE

RMSE

RMSE

RMSE

RMSE

 

VAR

0.71

0.71

0.27

0.72

0.75

0.60

BASMOD

0.81

0.82

0.65

0.82

0.81

0.76

 

 

Table 5 presents the results for CPI inflation. It shows that the forecast errors now are larger in BASMOD than in the VAR model. On the one and three year horizons forecasts are similar but on the two year horizon BASMOD performs less well. On average for all horizons and periods the forecast errors are about 20 percent lower in the VAR model than in BASMOD.

 

Recall that these evaluations use the most recent parameter estimates for both models and hence use information that was not available at the date of forecasting. In the evaluations with historical forecasts for BASMOD, Rixmod and the Riksbank no other information than was actually available at the time of the forecasting was used. To evaluate the effects of parameter updating I estimated BASMOD up to the date of forecasting and investigated how the updating affected the forecasts. Generally the forecasts improve significantly when parameters are updated, though not so much for CPI inflation and GDP growth. This casts some doubts on the comparison with the VAR above and reinforces the argument that the best evaluations are probably done by using actual forecasts.