Python數(shù)據(jù)正態(tài)性檢驗(yàn)實(shí)現(xiàn)過程
在做數(shù)據(jù)分析或者統(tǒng)計的時候,經(jīng)常需要進(jìn)行數(shù)據(jù)正態(tài)性的檢驗(yàn),因?yàn)楹芏嗉僭O(shè)都是基于正態(tài)分布的基礎(chǔ)之上的,例如:T檢驗(yàn)。
在Python中,主要有以下檢驗(yàn)正態(tài)性的方法:
1.scipy.stats.shapiro ——Shapiro-Wilk test,屬于專門用來做正態(tài)性檢驗(yàn)的模塊,其原假設(shè):樣本數(shù)據(jù)符合正態(tài)分布。
注:適用于小樣本。
其函數(shù)定位為:
def shapiro(x): ''' Perform the Shapiro-Wilk test for normality. The Shapiro-Wilk test tests the null hypothesis that the data was drawn from a normal distribution. Parameters ---------- x : array_like Array of sample data. Returns ------- W : float The test statistic. p-value : float The p-value for the hypothesis test.
x參數(shù)為樣本值序列,返回值中第一個為檢驗(yàn)統(tǒng)計量,第二個為P值,當(dāng)P值大于指定的顯著性水平,則接受原假設(shè)。
2.scipy.stats.kstest(K-S檢驗(yàn)):可以檢驗(yàn)多種分布,不止正態(tài)分布,其原假設(shè):數(shù)據(jù)符合正態(tài)分布。
其函數(shù)定義為:
def kstest(rvs, cdf, args=(), N=20, alternative=’two-sided’, mode=’approx’): ''' Perform the Kolmogorov-Smirnov test for goodness of fit. This performs a test of the distribution G(x) of an observed random variable against a given distribution F(x). Under the null hypothesis the two distributions are identical, G(x)=F(x). The alternative hypothesis can be either ’two-sided’ (default), ’less’ or ’greater’. The KS test is only valid for continuous distributions. Parameters ---------- rvs : str, array or callable If a string, it should be the name of a distribution in `scipy.stats`. If an array, it should be a 1-D array of observations of random variables. If a callable, it should be a function to generate random variables; it is required to have a keyword argument `size`. cdf : str or callable If a string, it should be the name of a distribution in `scipy.stats`. If `rvs` is a string then `cdf` can be False or the same as `rvs`. If a callable, that callable is used to calculate the cdf. args : tuple, sequence, optional Distribution parameters, used if `rvs` or `cdf` are strings. N : int, optional Sample size if `rvs` is string or callable. Default is 20. alternative : {’two-sided’, ’less’,’greater’}, optional Defines the alternative hypothesis (see explanation above). Default is ’two-sided’. mode : ’approx’ (default) or ’asymp’, optional Defines the distribution used for calculating the p-value. - ’approx’ : use approximation to exact distribution of test statistic - ’asymp’ : use asymptotic distribution of test statistic Returns ------- statistic : float KS test statistic, either D, D+ or D-. pvalue : float One-tailed or two-tailed p-value.
參數(shù)是:
rvs:待檢驗(yàn)數(shù)據(jù)。
cdf:檢驗(yàn)分布,例如’norm’,’expon’,’rayleigh’,’gamma’等分布,設(shè)置為’norm’時表示正態(tài)分布。
alternative:默認(rèn)為雙側(cè)檢驗(yàn),可以設(shè)置為’less’或’greater’作單側(cè)檢驗(yàn)。
model:’approx’(默認(rèn)值),表示使用檢驗(yàn)統(tǒng)計量的精確分布的近視值;’asymp’:使用檢驗(yàn)統(tǒng)計量的漸進(jìn)分布。
其返回值中第一個為統(tǒng)計量,第二個為P值。
3.scipy.stats.normaltest:正態(tài)性檢驗(yàn),其原假設(shè):樣本來自正態(tài)分布。
其函數(shù)定義為:
def normaltest(a, axis=0, nan_policy=’propagate’): ''' Test whether a sample differs from a normal distribution. This function tests the null hypothesis that a sample comes from a normal distribution. It is based on D’Agostino and Pearson’s [1]_, [2]_ test that combines skew and kurtosis to produce an omnibus test of normality. Parameters ---------- a : array_like The array containing the sample to be tested. axis : int or None, optional Axis along which to compute test. Default is 0. If None, compute over the whole array `a`. nan_policy : {’propagate’, ’raise’, ’omit’}, optional Defines how to handle when input contains nan. ’propagate’ returns nan, ’raise’ throws an error, ’omit’ performs the calculations ignoring nan values. Default is ’propagate’. Returns ------- statistic : float or array ``s^2 + k^2``, where ``s`` is the z-score returned by `skewtest` and ``k`` is the z-score returned by `kurtosistest`. pvalue : float or array A 2-sided chi squared probability for the hypothesis test.
其參數(shù):
axis=None 可以表示對整個數(shù)據(jù)做檢驗(yàn),默認(rèn)值是0。
nan_policy:當(dāng)輸入的數(shù)據(jù)中有nan時,’propagate’,返回空值;’raise’ 時,拋出錯誤;’omit’ 時,忽略空值。
其返回值中,第一個是統(tǒng)計量,第二個是P值。
4.scipy.stats.anderson:由 scipy.stats.kstest 改進(jìn)而來,用于檢驗(yàn)樣本是否屬于某一分布(正態(tài)分布、指數(shù)分布、logistic 或者 Gumbel等分布)
其函數(shù)定義為:
def anderson(x, dist=’norm’): ''' Anderson-Darling test for data coming from a particular distribution The Anderson-Darling tests the null hypothesis that a sample is drawn from a population that follows a particular distribution. For the Anderson-Darling test, the critical values depend on which distribution is being tested against. This function works for normal, exponential, logistic, or Gumbel (Extreme Value Type I) distributions. Parameters ---------- x : array_like array of sample data dist : {’norm’,’expon’,’logistic’,’gumbel’,’gumbel_l’, gumbel_r’, ’extreme1’}, optional the type of distribution to test against. The default is ’norm’ and ’extreme1’, ’gumbel_l’ and ’gumbel’ are synonyms. Returns ------- statistic : float The Anderson-Darling test statistic critical_values : list The critical values for this distribution significance_level : list The significance levels for the corresponding critical values in percents. The function returns critical values for a differing set of significance levels depending on the distribution that is being tested against.
其參數(shù):
x和dist分別表示樣本數(shù)據(jù)和分布。
返回值有三個,第一個表示統(tǒng)計值,第二個表示評價值,第三個是顯著性水平;評價值和顯著性水平對應(yīng)。
對于不同的分布,顯著性水平不一樣。
Critical values provided are for the following significance levels: normal/exponenential 15%, 10%, 5%, 2.5%, 1% logistic 25%, 10%, 5%, 2.5%, 1%, 0.5% Gumbel 25%, 10%, 5%, 2.5%, 1%
關(guān)于統(tǒng)計值與評價值的對比:當(dāng)統(tǒng)計值大于這些評價值時,表示在對應(yīng)的顯著性水平下,原假設(shè)被拒絕,即不屬于某分布。
If the returned statistic is larger than these critical values then for the corresponding significance level, the null hypothesis that the data come from the chosen distribution can be rejected.
5.skewtest 和kurtosistest 檢驗(yàn):用于檢驗(yàn)樣本的skew(偏度)和kurtosis(峰度)是否與正態(tài)分布一致,因?yàn)檎龖B(tài)分布的偏度=0,峰度=3。
偏度:偏度是樣本的標(biāo)準(zhǔn)三階中心矩。
峰度:峰度是樣本的標(biāo)準(zhǔn)四階中心矩。
6. 代碼如下:
import numpy as npfrom scipy import statsa = np.random.normal(0,2,50)b = np.linspace(0, 10, 100)# Shapiro-Wilk testS,p = stats.shapiro(a)print(’the shapiro test result is:’,S,’,’,p)# kstest(K-S檢驗(yàn))K,p = stats.kstest(a, ’norm’)print(K,p)# normaltestN,p = stats.normaltest(b)print(N,p)# Anderson-Darling testA,C,p = stats.anderson(b,dist=’norm’)print(A,C,p)
以上就是本文的全部內(nèi)容,希望對大家的學(xué)習(xí)有所幫助,也希望大家多多支持好吧啦網(wǎng)。
相關(guān)文章:
1. IntelliJ IDEA多屏后窗口不顯示問題解決方案2. docker compose idea CreateProcess error=2 系統(tǒng)找不到指定的文件的問題3. Vue 實(shí)現(xiàn)監(jiān)聽窗口關(guān)閉事件,并在窗口關(guān)閉前發(fā)送請求4. IntelliJ IDEA 2021.1 EAP 4 發(fā)布:字體粗細(xì)可調(diào)整Git commit template 支持5. 一文秒懂idea的git插件跟翻譯插件6. PHP 工程師面試的四個環(huán)節(jié)7. python與idea的集成的實(shí)現(xiàn)8. Android okhttp的啟動流程及源碼解析9. python基于OpenCV模板匹配識別圖片中的數(shù)字10. 使用idea開發(fā)javaWeb應(yīng)用程序的思路(實(shí)現(xiàn)用戶的增刪改查)
