Panel Data Analysis Tutorial - Interpreting the Results

Interpreting the Panel Data Results

The following results are from the OfficeStar Tutorial data set that loads automatically when you select the Tutorial link in the Enginius Dashboard and run with analysis parameters indicated in Running a Panel Data Analysis article. 

Data plots

The first set of outputs simply plot the data for each panel ID. Only the data for the first 15 panel IDs are plotted. These plots will show the distribution of the dependent variable values (Conversions) observed for each specific panel ID. If you need to see the plots for the other variables, re-order input data so that the panel IDs of interest to you appear first in the data set.

It may be useful to check the data for panel IDs with a large variance (e.g., “office supplies”) to make sure there are no data errors. Here, the large variance in the performance of the keyword “office supplies” could be because it is a generic keyword that might attract a broad audience only some of whom purchase from the company. Further people searching with this keyword are more likely to explore products from different competitors. Depending on the specific promotions being offered by competitors at a time when a specific campaign is being run, they may choose to purchase from different competitors in different time periods. 

Statistics Associated with the Selected Model

The first output is optional and will appear depending on which model is selected by the Automatic modeling option.  Here the model selected the Random-effects model as the most appropriate one for the OfficeStar data. The Pooled-OLS model was rejected (i.e., it was inferior to both the Fixed-effects model and the Random-effects model). Then the Hausman test was applied to check whether the Fixed effects model would be the most appropriate. Here that hypothesis is rejected because the significance level is much greater than 0.05. And, the preferred model is the Random effects model.  

The next set of results pertain to the estimated coefficients from the selected model. Here, the results of the Random effects model are summarized. Had the automatic options selected the Pooled-OLS or the Fixed-effects model, those results would be displayed instead.

This model has good fit (R-squared is significantly higher than 0). The results suggest that the number of Clicks and the total expenditures (Cost) have a positive influence on generating Conversions, even after accounting for the effects of unobserved characteristics associated with the keywords (Panel IDs). The significant positive coefficients are highlighted in green and the significant negative coefficients are highlighted in red. Interestingly, for OfficeStar, the number of Impressions generated by a keyword or the Avg Position at which the keyword advertising appeared do not seem to influence Conversions (if the ad appears at the top or the first position it has a position value of 1, in the second position, it will have a value equal to by 2, etc.).

In addition, for the Random-effects model, the output provides the estimated distribution of the random effects across all panel IDs. Here the distribution is a bit left-skewed. The two vertical lines correspond to the mean and standard deviation of the distribution. The random effects are the incremental effects on Conversions due to the unobserved characteristics associated with a Panel ID (Keyword).

Sizes of the random or fixed effects associated with each panel ID

The next set of outputs summarize the sizes of the fixed effects or the random effects depending on the model selected for analysis (this table will not appear for the Pooled-OLS model). The reported effects are the deviations for a specific panel ID from the overall average (equal to 0). 

Random Effects Example:

Fixed Effects Example:

We can see that the branded keyword “office star” has a strong positive impact on Conversions over and above the effects due to the number of Clicks garnered by that keyword, or the Cost (amount spent) on that Keyword. This is to be expected because those searching using a branded keyword are already familiar with the OfficeStar brand and may have favorable impressions about that brand.  On the other hand, the keyword “office star coupons” generates a negative effect on Conversions (after accounting for the effects of the other variables included in the panel regression model). This result may be surprising and should warrant closer examination. Perhaps this result occurs because those searching for coupons may be price sensitive and may be disappointed when they find that OfficeStar does not offer any discount coupons, or those searchers may find the prices of products sold by OfficeStar to be at a higher price point than what they expected.   

A related set of outputs plot the random or fixed effects ordering them from the highest to the lowest fixed effect based on the absolute magnitudes of these effects. When there are many panel IDs, these plots simplify interpretation by focusing attention on the most important effects.

Correlation matrix of coefficients

The final output contains the correlations amongst the coefficients of the independent variables. The values range from 1 for perfectly correlated coefficients to -1 for perfectly negatively correlated coefficients. A value of 0 means the two estimated coefficients are not correlated.

It is useful to check whether the correlations make sense, especially regarding potential multicollinearity, which results in the coefficient estimates being unreliable. Here, there is strong positive correlation between the independent variables Clicks and Cost, which can be computed to be equal to 0.92. This correlation makes sense – this data is from a paid search campaign where the costs are incurred through a “pay-per-click” model, which means the company incurs higher costs when there are more clicks. But the correlation between their coefficient estimates is negative. This result is somewhat surprising but is likely to be due to branded keywords costing less money, but generating higher clicks and conversions compared to unbranded keywords that cost more (i.e., generate more clicks) but result in fewer conversions, even if they generate more clicks. A similar process may be at play regarding the positive correlation between Avg Position and Cost -- this company is spending money on competitive keywords thereby incurring higher costs (i.e., the company needs to bid higher amounts for the same position) which do not necessarily result in a higher placement in the listing (note that the lower the Avg Position, the better is the placement of the ad on the paid listing).