The setting of the Confidence Level defines how secure we want to be to be to express a statistical test result.
When we for example claim that a certain difference of percentages or means is significant, or that a certain result 11% has a confidence interval at +/- 1,1 this does not mean that we can be 100% sure of this result, but we can be sure on a certain level.
Setting the Confidence level is set at 95% (most commonly used level in market research) this is an indication that we can be 95% sure that the result is significant or that we can be 95% sure that the true result in the population is within 9,9 -12,1%.
Most researchers use 90%, 95% or 99% Confidence Intervals (the most commonly used is 95% which is the default setting in OfficeReports).
In regards to Sample Size, it is simply the larger the better, as a larger sample size will more accurately reflect/represent the population. The larger the sample size means the smaller the confidence interval needs to be set at for a given confidence level.
When adding confidence interval to a table OfficeReports by default assume that the sample represents a large population and the sample represents less than 10% of the population. If the sample represents a significant proportion of the population (> 10%) we can enter the Population Size in order to adjust the test formula. As a consequence, the confidence interval reduces when the sample proportion get high.
We can add confidence intervals to percentages in frequency tables.
The output is an extra column displaying the confidence level, based on the confidence level defined in ‘SignificanceàConfidence Interval’ (default= 95%).
For example, the confidence level to 23% is 3,6 which means that we can be 95% sure that between 19,4 – 26,6% of the population strongly agree with the statement ‘I understand my benefit plan ‘.
When ‘Mean’ is defined as a part of a frequency table ‘Mean Confidence’ can be added to the frequency table:
In this example, we can be 95% sure that the mean in the population is between 3,0 – 3,2: