Full Code of fengdu78/Data-Science-Notes for AI

Repository: fengdu78/Data-Science-Notes
Branch: master
Commit: ad7356537d9b
Files: 186
Total size: 24.5 MB

Directory structure:
gitextract_d5cmywkc/

├── 0.math/
│   ├── 0.basic/
│   │   ├── README.md
│   │   └── markdown/
│   │       └── README.md
│   ├── 1.CS229/
│   │   ├── README.md
│   │   └── markdown/
│   │       ├── 1.CS229-LinearAlgebra.md
│   │       ├── 2.CS229-Prob.md
│   │       └── README.md
│   └── README.md
├── 1.python-basic/
│   ├── Python-100/
│   │   ├── README.md
│   │   ├── Status/
│   │   │   ├── Day 1.md
│   │   │   ├── Day 2.md
│   │   │   ├── Day 3.md
│   │   │   ├── Day 4.md
│   │   │   ├── Day 5.md
│   │   │   ├── Day 6.md
│   │   │   ├── Day 7.md
│   │   │   ├── Day 8.md
│   │   │   ├── Day 9.md
│   │   │   ├── Day_10.md
│   │   │   ├── Day_11.md
│   │   │   ├── Day_12.md
│   │   │   ├── Day_13.md
│   │   │   ├── Day_14.md
│   │   │   ├── Day_15.md
│   │   │   ├── Day_16.md
│   │   │   ├── Day_17.md
│   │   │   ├── Day_18.md
│   │   │   ├── Day_19.md
│   │   │   ├── Day_20.md
│   │   │   ├── Day_21.md
│   │   │   ├── Day_22.md
│   │   │   ├── Day_23.md
│   │   │   └── Day_24.md
│   │   └── notebooks/
│   │       ├── Day_01.ipynb
│   │       ├── Day_02.ipynb
│   │       ├── Day_03.ipynb
│   │       ├── Day_04.ipynb
│   │       ├── Day_05.ipynb
│   │       ├── Day_06.ipynb
│   │       ├── Day_07.ipynb
│   │       ├── Day_08.ipynb
│   │       ├── Day_09.ipynb
│   │       ├── Day_10.ipynb
│   │       ├── Day_11.ipynb
│   │       ├── Day_12.ipynb
│   │       ├── Day_13.ipynb
│   │       ├── Day_14.ipynb
│   │       ├── Day_15.ipynb
│   │       ├── Day_16.ipynb
│   │       ├── Day_17.ipynb
│   │       ├── Day_18.ipynb
│   │       ├── Day_19.ipynb
│   │       ├── Day_20.ipynb
│   │       ├── Day_21.ipynb
│   │       ├── Day_22.ipynb
│   │       ├── Day_23.ipynb
│   │       └── Day_24.ipynb
│   ├── Python_Basic.ipynb
│   ├── a.txt
│   ├── ly.txt
│   ├── ly2.txt
│   ├── test
│   ├── 两天入门python-第一天.ipynb
│   └── 两天入门python-第二天.ipynb
├── 2.numpy/
│   ├── 1.numpy-beginner.ipynb
│   ├── 2.numpy.ipynb
│   ├── README.md
│   ├── numpy-100/
│   │   ├── 100_Numpy_exercises.ipynb
│   │   ├── 100_Numpy_exercises_no_solution.ipynb
│   │   ├── 100_Numpy_exercises_with_hint.ipynb
│   │   └── README.md
│   └── numpy_exercises/
│       ├── 10_Random_sampling.ipynb
│       ├── 10_Random_sampling_Solutions.ipynb
│       ├── 11_Set_routines.ipynb
│       ├── 11_Set_routines_Solutions.ipynb
│       ├── 12_Sorting_searching_and_counting.ipynb
│       ├── 12_Sorting_searching_and_counting_Solutions.ipynb
│       ├── 13_Statistics.ipynb
│       ├── 13_Statistics_solutions.ipynb
│       ├── 1_Array_creation_routines.ipynb
│       ├── 1_Array_creation_routines_Solution.ipynb
│       ├── 2_Array_manipulation_routines.ipynb
│       ├── 2_Array_manipulation_routines_Solutions.ipynb
│       ├── 3_String_operations.ipynb
│       ├── 3_String_operations_solutions.ipynb
│       ├── 4_Numpy-specific_help_functions.ipynb
│       ├── 4_Numpy-specific_help_functions_Solutions.ipynb
│       ├── 5_Input_and_Output.ipynb
│       ├── 5_Input_and_Output_Solutions.ipynb
│       ├── 6_Linear_algebra.ipynb
│       ├── 6_Linear_algebra_Solutions.ipynb
│       ├── 7_Discrete_Fourier_Transform.ipynb
│       ├── 7_Discrete_Fourier_Transform_solutions.ipynb
│       ├── 8_Logic_functions.ipynb
│       ├── 8_Logic_functions_Solutions.ipynb
│       ├── 9_Mathematical_functions.ipynb
│       ├── 9_Mathematical_functions_solutions.ipynb
│       └── README.md
├── 3.pandas/
│   ├── 1.10-Minutes-to-pandas/
│   │   ├── 10 Minutes to pandas.ipynb
│   │   ├── README.md
│   │   └── data/
│   │       ├── foo.csv
│   │       ├── foo.h5
│   │       └── foo.xlsx
│   ├── 2.Pandas_Exercises/
│   │   └── README.md
│   ├── 3.pandas_beginner/
│   │   ├── README.md
│   │   ├── pandas_beginner.ipynb
│   │   └── student.csv
│   ├── 4.Pandas50/
│   │   ├── Pandas50.ipynb
│   │   └── README.md
│   └── README.md
├── 4.scipy/
│   ├── 1.scipy-intro.ipynb
│   ├── 10.scipy-spatial.ipynb
│   ├── 2.scipy-optimize.ipynb
│   ├── 3.scipy-linalg.ipynb
│   ├── 4.scipy-stats.ipynb
│   ├── 5.scipy-integrate.ipynb
│   ├── 6.scipy-signal.ipynb
│   ├── 7.scipy-interpolate.ipynb
│   ├── 8.scipy-sparse.ipynb
│   ├── 9.scipy-ndimage.ipynb
│   └── README.md
├── 5.data-visualization/
│   ├── 1.matplotlib/
│   │   ├── examples/
│   │   │   └── spx.csv
│   │   ├── matplotlib-beginner.ipynb
│   │   ├── matplotlib50/
│   │   │   ├── Pokemon.csv
│   │   │   └── matplotlib50.ipynb
│   │   └── plot_distribution/
│   │       ├── README.md
│   │       └── plot_distribution.ipynb
│   ├── 2.seaborn/
│   │   ├── README.md
│   │   └── Searborn.ipynb
│   └── README.md
├── 6.scikit-learn/
│   ├── .gitignore
│   ├── LICENSE
│   ├── README.md
│   ├── check_environment.py
│   ├── data/
│   │   ├── README.md
│   │   ├── adult_openml.csv
│   │   └── titanic_openml.csv
│   ├── environment.yml
│   ├── notebook.ipynb
│   └── solutions/
│       ├── 01_1_solutions.py
│       ├── 01_2_solutions.py
│       ├── 01_3_solutions.py
│       ├── 01_4_solutions.py
│       ├── 01_5_solutions.py
│       ├── 01_solutions.py
│       ├── 02_solutions.py
│       ├── 03_solutions.py
│       ├── 04_solutions.py
│       ├── 05_1_solutions.py
│       ├── 05_2_solutions.py
│       ├── 05_3_solutions.py
│       ├── 05_4_solutions.py
│       ├── 05_5_solutions.py
│       ├── 05_6_solutions.py
│       ├── 05_7_solutions.py
│       └── 05_solutions.py
├── 7.machine-learning/
│   └── README.md
├── 8.deep-learning/
│   ├── Deep-Learning-Papers-Reading-Roadmap/
│   │   └── README.md
│   ├── PyTorch_beginner/
│   │   ├── 1.Tensors.ipynb
│   │   ├── 2.Autograd自动求导.ipynb
│   │   ├── 3.神经网络.ipynb
│   │   ├── 4.训练一个分类器.ipynb
│   │   └── README.md
│   ├── README.md
│   └── word2vec/
│       ├── README.md
│       └── Visualizing_embeddings.ipynb
├── 9.feature-engineering/
│   ├── 1.引言.ipynb
│   ├── 2.简单数字的奇特技巧.ipynb
│   ├── 3.文本数据.ipynb
│   ├── 4.特征缩放的效果：从词袋到_TF-IDF.ipynb
│   ├── 5.类别特征.ipynb
│   ├── 6.降维：用_PCA_压缩数据集.ipynb
│   ├── 7.非线性特征提取和模型堆叠.ipynb
│   ├── 8.自动化特征提取器：图像特征提取和深度学习.ipynb
│   ├── 9.回到特征：将它们放到一起.ipynb
│   ├── FeatureSelectorUsage/
│   │   ├── FeatureSelectorUsage.ipynb
│   │   ├── README.md
│   │   ├── data/
│   │   │   ├── AirQualityUCI.csv
│   │   │   ├── caravan-insurance-challenge.csv
│   │   │   └── credit_example.csv
│   │   └── feature_selector/
│   │       ├── __init__.py
│   │       └── feature_selector.py
│   ├── README.md
│   ├── data/
│   │   └── README.md
│   ├── images/
│   │   └── Appendix/
│   │       └── pic
│   └── 附录.线性模型和线性代数基础.ipynb
└── README.md

================================================
FILE CONTENTS
================================================

================================================
FILE: 0.math/0.basic/README.md
================================================

大学数学基础知识
------------------

数据科学需要一定的数学基础，但仅仅做应用的话，如果时间不多，不用学太深，了解基本公式即可，遇到问题再查吧。

以下是以前考研考博时候的数学笔记，难度应该在本科3年级左右。 

markdown文件夹是文件的markdown代码，内容与pdf一致，是为了方便研究者写论文或者文章使用。



================================================
FILE: 0.math/0.basic/markdown/README.md
================================================

数学基础知识
------------------

[TOC]

数据科学需要一定的数学基础，但仅仅做应用的话，如果时间不多，不用学太深，了解基本公式即可，遇到问题再查吧。

以下是以前考研考博时候的数学笔记，难度应该在本科3年级左右。 

### 高等数学

**1.导数定义：**

导数和微分的概念

$f'({{x}_{0}})=\underset{\Delta x\to 0}{\mathop{\lim }}\,\frac{f({{x}_{0}}+\Delta x)-f({{x}_{0}})}{\Delta x}$    （1）

或者：

$f'({{x}_{0}})=\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,\frac{f(x)-f({{x}_{0}})}{x-{{x}_{0}}}$           （2）

**2.左右导数导数的几何意义和物理意义**

函数$f(x)$在$x_0$处的左、右导数分别定义为：

左导数：${{{f}'}_{-}}({{x}_{0}})=\underset{\Delta x\to {{0}^{-}}}{\mathop{\lim }}\,\frac{f({{x}_{0}}+\Delta x)-f({{x}_{0}})}{\Delta x}=\underset{x\to x_{0}^{-}}{\mathop{\lim }}\,\frac{f(x)-f({{x}_{0}})}{x-{{x}_{0}}},(x={{x}_{0}}+\Delta x)$

右导数：${{{f}'}_{+}}({{x}_{0}})=\underset{\Delta x\to {{0}^{+}}}{\mathop{\lim }}\,\frac{f({{x}_{0}}+\Delta x)-f({{x}_{0}})}{\Delta x}=\underset{x\to x_{0}^{+}}{\mathop{\lim }}\,\frac{f(x)-f({{x}_{0}})}{x-{{x}_{0}}}$

**3.函数的可导性与连续性之间的关系**

**Th1:** 函数$f(x)$在$x_0$处可微$\Leftrightarrow f(x)$在$x_0$处可导

**Th2:** 若函数在点$x_0$处可导，则$y=f(x)$在点$x_0$处连续，反之则不成立。即函数连续不一定可导。

**Th3:** ${f}'({{x}_{0}})$存在$\Leftrightarrow {{{f}'}_{-}}({{x}_{0}})={{{f}'}_{+}}({{x}_{0}})$

**4.平面曲线的切线和法线**

切线方程 : $y-{{y}_{0}}=f'({{x}_{0}})(x-{{x}_{0}})$
法线方程：$y-{{y}_{0}}=-\frac{1}{f'({{x}_{0}})}(x-{{x}_{0}}),f'({{x}_{0}})\ne 0$

**5.四则运算法则**
设函数$u=u(x)，v=v(x)$]在点$x$可导则
(1) $(u\pm v{)}'={u}'\pm {v}'$       $d(u\pm v)=du\pm dv$
(2)$(uv{)}'=u{v}'+v{u}'$        $d(uv)=udv+vdu$
(3) $(\frac{u}{v}{)}'=\frac{v{u}'-u{v}'}{{{v}^{2}}}(v\ne 0)$       $d(\frac{u}{v})=\frac{vdu-udv}{{{v}^{2}}}$

**6.基本导数与微分表**
(1) $y=c$（常数）       ${y}'=0$          $dy=0$
(2) $y={{x}^{\alpha }}$($\alpha $为实数)    ${y}'=\alpha {{x}^{\alpha -1}}$      $dy=\alpha {{x}^{\alpha -1}}dx$
(3) $y={{a}^{x}}$      ${y}'={{a}^{x}}\ln a$         $dy={{a}^{x}}\ln adx$
  特例:   $({{{e}}^{x}}{)}'={{{e}}^{x}}$             $d({{{e}}^{x}})={{{e}}^{x}}dx$

(4) $y={{\log }_{a}}x$   ${y}'=\frac{1}{x\ln a}$           

$dy=\frac{1}{x\ln a}dx$
  特例:$y=\ln x$                      $(\ln x{)}'=\frac{1}{x}$       $d(\ln x)=\frac{1}{x}dx$

(5) $y=\sin x$         

${y}'=\cos x$        $d(\sin x)=\cos xdx$

(6) $y=\cos x$      

${y}'=-\sin x$       $d(\cos x)=-\sin xdx$

(7) $y=\tan x$  

${y}'=\frac{1}{{{\cos }^{2}}x}={{\sec }^{2}}x$  $d(\tan x)={{\sec }^{2}}xdx$
(8) $y=\cot x$ ${y}'=-\frac{1}{{{\sin }^{2}}x}=-{{\csc }^{2}}x$  $d(\cot x)=-{{\csc }^{2}}xdx$
(9) $y=\sec x$ ${y}'=\sec x\tan x$     

 $d(\sec x)=\sec x\tan xdx$
(10) $y=\csc x$ ${y}'=-\csc x\cot x$    

$d(\csc x)=-\csc x\cot xdx$
(11) $y=\arcsin x$  

${y}'=\frac{1}{\sqrt{1-{{x}^{2}}}}$   

$d(\arcsin x)=\frac{1}{\sqrt{1-{{x}^{2}}}}dx$
(12) $y=\arccos x$ 

${y}'=-\frac{1}{\sqrt{1-{{x}^{2}}}}$     $d(\arccos x)=-\frac{1}{\sqrt{1-{{x}^{2}}}}dx$

(13) $y=\arctan x$ 

${y}'=\frac{1}{1+{{x}^{2}}}$     $d(\arctan x)=\frac{1}{1+{{x}^{2}}}dx$

(14) $y=\operatorname{arc}\cot x$      

${y}'=-\frac{1}{1+{{x}^{2}}}$   

$d(\operatorname{arc}\cot x)=-\frac{1}{1+{{x}^{2}}}dx$
(15) $y=shx$    

${y}'=chx$       $d(shx)=chxdx$

(16) $y=chx$    

${y}'=shx$       $d(chx)=shxdx$

**7.复合函数，反函数，隐函数以及参数方程所确定的函数的微分法**

(1) 反函数的运算法则: 设$y=f(x)$在点$x$的某邻域内单调连续，在点$x$处可导且${f}'(x)\ne 0$，则其反函数在点$x$所对应的$y$处可导，并且有$\frac{dy}{dx}=\frac{1}{\frac{dx}{dy}}$
(2) 复合函数的运算法则:若$\mu =\varphi (x)$在点$x$可导,而$y=f(\mu )$在对应点$\mu $($\mu =\varphi (x)$)可导,则复合函数$y=f(\varphi (x))$在点$x$可导,且${y}'={f}'(\mu )\cdot {\varphi }'(x)$
(3) 隐函数导数$\frac{dy}{dx}$的求法一般有三种方法：
1)方程两边对$x$求导，要记住$y$是$x$的函数，则$y$的函数是$x$的复合函数.例如$\frac{1}{y}$，${{y}^{2}}$，$ln y$，${{{e}}^{y}}$等均是$x$的复合函数.
对$x$求导应按复合函数连锁法则做.
2)公式法.由$F(x,y)=0$知 $\frac{dy}{dx}=-\frac{{{{{F}'}}_{x}}(x,y)}{{{{{F}'}}_{y}}(x,y)}$,其中，${{{F}'}_{x}}(x,y)$，
${{{F}'}_{y}}(x,y)$分别表示$F(x,y)$对$x$和$y$的偏导数
3)利用微分形式不变性

**8.常用高阶导数公式**

（1）$({{a}^{x}}){{\,}^{(n)}}={{a}^{x}}{{\ln }^{n}}a\quad (a>{0})\quad \quad ({{{e}}^{x}}){{\,}^{(n)}}={e}{{\,}^{x}}$
（2）$(\sin kx{)}{{\,}^{(n)}}={{k}^{n}}\sin (kx+n\cdot \frac{\pi }{{2}})$
（3）$(\cos kx{)}{{\,}^{(n)}}={{k}^{n}}\cos (kx+n\cdot \frac{\pi }{{2}})$
（4）$({{x}^{m}}){{\,}^{(n)}}=m(m-1)\cdots (m-n+1){{x}^{m-n}}$
（5）$(\ln x){{\,}^{(n)}}={{(-{1})}^{(n-{1})}}\frac{(n-{1})!}{{{x}^{n}}}$
（6）莱布尼兹公式：若$u(x)\,,v(x)$均$n$阶可导，则
 ${{(uv)}^{(n)}}=\sum\limits_{i={0}}^{n}{c_{n}^{i}{{u}^{(i)}}{{v}^{(n-i)}}}$，其中${{u}^{({0})}}=u$，${{v}^{({0})}}=v$

**9.微分中值定理，泰勒公式**

**Th1:**(费马定理)

若函数$f(x)$满足条件：
(1)函数$f(x)$在${{x}_{0}}$的某邻域内有定义，并且在此邻域内恒有
$f(x)\le f({{x}_{0}})$或$f(x)\ge f({{x}_{0}})$,

(2) $f(x)$在${{x}_{0}}$处可导,则有 ${f}'({{x}_{0}})=0$

**Th2:**(罗尔定理) 

设函数$f(x)$满足条件：
(1)在闭区间$[a,b]$上连续；

(2)在$(a,b)$内可导；

(3)$f(a)=f(b)$；

则在$(a,b)$内一存在个$\xi $，使  ${f}'(\xi )=0$
**Th3:** (拉格朗日中值定理) 

设函数$f(x)$满足条件：
(1)在$[a,b]$上连续；

(2)在$(a,b)$内可导；

则在$(a,b)$内一存在个$\xi $，使  $\frac{f(b)-f(a)}{b-a}={f}'(\xi )$

**Th4:** (柯西中值定理)

 设函数$f(x)$，$g(x)$满足条件：
(1) 在$[a,b]$上连续；

(2) 在$(a,b)$内可导且${f}'(x)$，${g}'(x)$均存在，且${g}'(x)\ne 0$

则在$(a,b)$内存在一个$\xi $，使  $\frac{f(b)-f(a)}{g(b)-g(a)}=\frac{{f}'(\xi )}{{g}'(\xi )}$

**10.洛必达法则**
法则Ⅰ ($\frac{0}{0}$型)
设函数$f\left( x \right),g\left( x \right)$满足条件：
 $\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,f\left( x \right)=0,\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,g\left( x \right)=0$; 

$f\left( x \right),g\left( x \right)$在${{x}_{0}}$的邻域内可导，(在${{x}_{0}}$处可除外)且${g}'\left( x \right)\ne 0$;

$\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,\frac{{f}'\left( x \right)}{{g}'\left( x \right)}$存在(或$\infty $)。

则:
$\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,\frac{f\left( x \right)}{g\left( x \right)}=\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,\frac{{f}'\left( x \right)}{{g}'\left( x \right)}$。
法则${{I}'}$ ($\frac{0}{0}$型)设函数$f\left( x \right),g\left( x \right)$满足条件：
$\underset{x\to \infty }{\mathop{\lim }}\,f\left( x \right)=0,\underset{x\to \infty }{\mathop{\lim }}\,g\left( x \right)=0$;

存在一个$X>0$,当$\left| x \right|>X$时,$f\left( x \right),g\left( x \right)$可导,且${g}'\left( x \right)\ne 0$;$\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,\frac{{f}'\left( x \right)}{{g}'\left( x \right)}$存在(或$\infty $)。

则:
$\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,\frac{f\left( x \right)}{g\left( x \right)}=\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,\frac{{f}'\left( x \right)}{{g}'\left( x \right)}$
法则Ⅱ($\frac{\infty }{\infty }$型) 设函数$f\left( x \right),g\left( x \right)$满足条件：
$\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,f\left( x \right)=\infty ,\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,g\left( x \right)=\infty $;   $f\left( x \right),g\left( x \right)$在${{x}_{0}}$ 的邻域内可导(在${{x}_{0}}$处可除外)且${g}'\left( x \right)\ne 0$;$\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,\frac{{f}'\left( x \right)}{{g}'\left( x \right)}$存在(或$\infty $)。则
$\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,\frac{f\left( x \right)}{g\left( x \right)}=\underset{x\to {{x}_{0}}}{\mathop{\lim }}\,\frac{{f}'\left( x \right)}{{g}'\left( x \right)}.$同理法则${I{I}'}$($\frac{\infty }{\infty }$型)仿法则${{I}'}$可写出。

**11.泰勒公式**

设函数$f(x)$在点${{x}_{0}}$处的某邻域内具有$n+1$阶导数，则对该邻域内异于${{x}_{0}}$的任意点$x$，在${{x}_{0}}$与$x$之间至少存在
一个$\xi $，使得：
$f(x)=f({{x}_{0}})+{f}'({{x}_{0}})(x-{{x}_{0}})+\frac{1}{2!}{f}''({{x}_{0}}){{(x-{{x}_{0}})}^{2}}+\cdots $ 
$+\frac{{{f}^{(n)}}({{x}_{0}})}{n!}{{(x-{{x}_{0}})}^{n}}+{{R}_{n}}(x)$
 其中 ${{R}_{n}}(x)=\frac{{{f}^{(n+1)}}(\xi )}{(n+1)!}{{(x-{{x}_{0}})}^{n+1}}$称为$f(x)$在点${{x}_{0}}$处的$n$阶泰勒余项。

令${{x}_{0}}=0$，则$n$阶泰勒公式
$f(x)=f(0)+{f}'(0)x+\frac{1}{2!}{f}''(0){{x}^{2}}+\cdots +\frac{{{f}^{(n)}}(0)}{n!}{{x}^{n}}+{{R}_{n}}(x)$……(1)
其中 ${{R}_{n}}(x)=\frac{{{f}^{(n+1)}}(\xi )}{(n+1)!}{{x}^{n+1}}$，$\xi $在0与$x$之间.(1)式称为麦克劳林公式

**常用五种函数在${{x}_{0}}=0$处的泰勒公式**

(1) ${{{e}}^{x}}=1+x+\frac{1}{2!}{{x}^{2}}+\cdots +\frac{1}{n!}{{x}^{n}}+\frac{{{x}^{n+1}}}{(n+1)!}{{e}^{\xi }}$ 

或 $=1+x+\frac{1}{2!}{{x}^{2}}+\cdots +\frac{1}{n!}{{x}^{n}}+o({{x}^{n}})$

(2) $\sin x=x-\frac{1}{3!}{{x}^{3}}+\cdots +\frac{{{x}^{n}}}{n!}\sin \frac{n\pi }{2}+\frac{{{x}^{n+1}}}{(n+1)!}\sin (\xi +\frac{n+1}{2}\pi )$

或  $=x-\frac{1}{3!}{{x}^{3}}+\cdots +\frac{{{x}^{n}}}{n!}\sin \frac{n\pi }{2}+o({{x}^{n}})$

(3) $\cos x=1-\frac{1}{2!}{{x}^{2}}+\cdots +\frac{{{x}^{n}}}{n!}\cos \frac{n\pi }{2}+\frac{{{x}^{n+1}}}{(n+1)!}\cos (\xi +\frac{n+1}{2}\pi )$

或   $=1-\frac{1}{2!}{{x}^{2}}+\cdots +\frac{{{x}^{n}}}{n!}\cos \frac{n\pi }{2}+o({{x}^{n}})$

(4) $\ln (1+x)=x-\frac{1}{2}{{x}^{2}}+\frac{1}{3}{{x}^{3}}-\cdots +{{(-1)}^{n-1}}\frac{{{x}^{n}}}{n}+\frac{{{(-1)}^{n}}{{x}^{n+1}}}{(n+1){{(1+\xi )}^{n+1}}}$

或      $=x-\frac{1}{2}{{x}^{2}}+\frac{1}{3}{{x}^{3}}-\cdots +{{(-1)}^{n-1}}\frac{{{x}^{n}}}{n}+o({{x}^{n}})$

(5) ${{(1+x)}^{m}}=1+mx+\frac{m(m-1)}{2!}{{x}^{2}}+\cdots +\frac{m(m-1)\cdots (m-n+1)}{n!}{{x}^{n}}$ 
$+\frac{m(m-1)\cdots (m-n+1)}{(n+1)!}{{x}^{n+1}}{{(1+\xi )}^{m-n-1}}$ 

或 ${{(1+x)}^{m}}=1+mx+\frac{m(m-1)}{2!}{{x}^{2}}+\cdots $ $+\frac{m(m-1)\cdots (m-n+1)}{n!}{{x}^{n}}+o({{x}^{n}})$

**12.函数单调性的判断**
**Th1:**  设函数$f(x)$在$(a,b)$区间内可导，如果对$\forall x\in (a,b)$，都有$f\,'(x)>0$（或$f\,'(x)<0$），则函数$f(x)$在$(a,b)$内是单调增加的（或单调减少）

**Th2:** （取极值的必要条件）设函数$f(x)$在${{x}_{0}}$处可导，且在${{x}_{0}}$处取极值，则$f\,'({{x}_{0}})=0$。

**Th3:** （取极值的第一充分条件）设函数$f(x)$在${{x}_{0}}$的某一邻域内可微，且$f\,'({{x}_{0}})=0$（或$f(x)$在${{x}_{0}}$处连续，但$f\,'({{x}_{0}})$不存在。）
(1)若当$x$经过${{x}_{0}}$时，$f\,'(x)$由“+”变“-”，则$f({{x}_{0}})$为极大值；
(2)若当$x$经过${{x}_{0}}$时，$f\,'(x)$由“-”变“+”，则$f({{x}_{0}})$为极小值；
(3)若$f\,'(x)$经过$x={{x}_{0}}$的两侧不变号，则$f({{x}_{0}})$不是极值。

**Th4:** (取极值的第二充分条件)设$f(x)$在点${{x}_{0}}$处有$f''(x)\ne 0$，且$f\,'({{x}_{0}})=0$，则 当$f'\,'({{x}_{0}})<0$时，$f({{x}_{0}})$为极大值；
当$f'\,'({{x}_{0}})>0$时，$f({{x}_{0}})$为极小值。
注：如果$f'\,'({{x}_{0}})<0$，此方法失效。

**13.渐近线的求法**
(1)水平渐近线   若$\underset{x\to +\infty }{\mathop{\lim }}\,f(x)=b$，或$\underset{x\to -\infty }{\mathop{\lim }}\,f(x)=b$，则

$y=b$称为函数$y=f(x)$的水平渐近线。

(2)铅直渐近线   若$\underset{x\to x_{0}^{-}}{\mathop{\lim }}\,f(x)=\infty $，或$\underset{x\to x_{0}^{+}}{\mathop{\lim }}\,f(x)=\infty $，则

$x={{x}_{0}}$称为$y=f(x)$的铅直渐近线。

(3)斜渐近线   若$a=\underset{x\to \infty }{\mathop{\lim }}\,\frac{f(x)}{x},\quad b=\underset{x\to \infty }{\mathop{\lim }}\,[f(x)-ax]$，则
$y=ax+b$称为$y=f(x)$的斜渐近线。

**14.函数凹凸性的判断**
**Th1:** (凹凸性的判别定理）若在I上$f''(x)<0$（或$f''(x)>0$），则$f(x)$在I上是凸的（或凹的）。

**Th2:** (拐点的判别定理1)若在${{x}_{0}}$处$f''(x)=0$，（或$f''(x)$不存在），当$x$变动经过${{x}_{0}}$时，$f''(x)$变号，则$({{x}_{0}},f({{x}_{0}}))$为拐点。

**Th3:** (拐点的判别定理2)设$f(x)$在${{x}_{0}}$点的某邻域内有三阶导数，且$f''(x)=0$，$f'''(x)\ne 0$，则$({{x}_{0}},f({{x}_{0}}))$为拐点。

**15.弧微分**

$dS=\sqrt{1+y{{'}^{2}}}dx$

**16.曲率**

曲线$y=f(x)$在点$(x,y)$处的曲率$k=\frac{\left| y'' \right|}{{{(1+y{{'}^{2}})}^{\tfrac{3}{2}}}}$。
对于参数方程$\left\{ \begin{align}  & x=\varphi (t) \\  & y=\psi (t) \\ \end{align} \right.,$$k=\frac{\left| \varphi '(t)\psi ''(t)-\varphi ''(t)\psi '(t) \right|}{{{[\varphi {{'}^{2}}(t)+\psi {{'}^{2}}(t)]}^{\tfrac{3}{2}}}}$。

**17.曲率半径**

曲线在点$M$处的曲率$k(k\ne 0)$与曲线在点$M$处的曲率半径$\rho $有如下关系：$\rho =\frac{1}{k}$。

### 线性代数

#### 行列式

**1.行列式按行（列）展开定理**

(1) 设$A = ( a_{{ij}} )_{n \times n}$，则：$a_{i1}A_{j1} +a_{i2}A_{j2} + \cdots + a_{{in}}A_{{jn}} = \begin{cases}|A|,i=j\\ 0,i \neq j\end{cases}$


或$a_{1i}A_{1j} + a_{2i}A_{2j} + \cdots + a_{{ni}}A_{{nj}} = \begin{cases}|A|,i=j\\ 0,i \neq j\end{cases}$即 $AA^{*} = A^{*}A = \left| A \right|E,$其中：$A^{*} = \begin{pmatrix} A_{11} & A_{12} & \ldots & A_{1n} \\ A_{21} & A_{22} & \ldots & A_{2n} \\ \ldots & \ldots & \ldots & \ldots \\ A_{n1} & A_{n2} & \ldots & A_{{nn}} \\ \end{pmatrix} = (A_{{ji}}) = {(A_{{ij}})}^{T}$

$D_{n} = \begin{vmatrix} 1 & 1 & \ldots & 1 \\ x_{1} & x_{2} & \ldots & x_{n} \\ \ldots & \ldots & \ldots & \ldots \\ x_{1}^{n - 1} & x_{2}^{n - 1} & \ldots & x_{n}^{n - 1} \\ \end{vmatrix} = \prod_{1 \leq j < i \leq n}^{}\,(x_{i} - x_{j})$

(2) 设$A,B$为$n$阶方阵，则$\left| {AB} \right| = \left| A \right|\left| B \right| = \left| B \right|\left| A \right| = \left| {BA} \right|$，但$\left| A \pm B \right| = \left| A \right| \pm \left| B \right|$不一定成立。

(3) $\left| {kA} \right| = k^{n}\left| A \right|$,$A$为$n$阶方阵。

(4) 设$A$为$n$阶方阵，$|A^{T}| = |A|;|A^{- 1}| = |A|^{- 1}$（若$A$可逆），$|A^{*}| = |A|^{n - 1}$

$n \geq 2$

(5) $\left| \begin{matrix}  & {A\quad O} \\  & {O\quad B} \\ \end{matrix} \right| = \left| \begin{matrix}  & {A\quad C} \\  & {O\quad B} \\ \end{matrix} \right| = \left| \begin{matrix}  & {A\quad O} \\  & {C\quad B} \\ \end{matrix} \right| =| A||B|$
，$A,B$为方阵，但$\left| \begin{matrix} {O} & A_{m \times m} \\  B_{n \times n} & { O} \\ \end{matrix} \right| = ({- 1)}^{{mn}}|A||B|$ 。

(6) 范德蒙行列式$D_{n} = \begin{vmatrix} 1 & 1 & \ldots & 1 \\ x_{1} & x_{2} & \ldots & x_{n} \\ \ldots & \ldots & \ldots & \ldots \\ x_{1}^{n - 1} & x_{2}^{n 1} & \ldots & x_{n}^{n - 1} \\ \end{vmatrix} =  \prod_{1 \leq j < i \leq n}^{}\,(x_{i} - x_{j})$

设$A$是$n$阶方阵，$\lambda_{i}(i = 1,2\cdots,n)$是$A$的$n$个特征值，则
$|A| = \prod_{i = 1}^{n}\lambda_{i}$

#### 矩阵

矩阵：$m \times n$个数$a_{{ij}}$排成$m$行$n$列的表格$\begin{bmatrix}  a_{11}\quad a_{12}\quad\cdots\quad a_{1n} \\ a_{21}\quad a_{22}\quad\cdots\quad a_{2n} \\ \quad\cdots\cdots\cdots\cdots\cdots \\  a_{m1}\quad a_{m2}\quad\cdots\quad a_{{mn}} \\ \end{bmatrix}$ 称为矩阵，简记为$A$，或者$\left( a_{{ij}} \right)_{m \times n}$ 。若$m = n$，则称$A$是$n$阶矩阵或$n$阶方阵。

**矩阵的线性运算**

**1.矩阵的加法**

设$A = (a_{{ij}}),B = (b_{{ij}})$是两个$m \times n$矩阵，则$m \times n$ 矩阵$C = c_{{ij}}) = a_{{ij}} + b_{{ij}}$称为矩阵$A$与$B$的和，记为$A + B = C$ 。

**2.矩阵的数乘**

设$A = (a_{{ij}})$是$m \times n$矩阵，$k$是一个常数，则$m \times n$矩阵$(ka_{{ij}})$称为数$k$与矩阵$A$的数乘，记为${kA}$。

**3.矩阵的乘法**

设$A = (a_{{ij}})$是$m \times n$矩阵，$B = (b_{{ij}})$是$n \times s$矩阵，那么$m \times s$矩阵$C = (c_{{ij}})$，其中$c_{{ij}} = a_{i1}b_{1j} + a_{i2}b_{2j} + \cdots + a_{{in}}b_{{nj}} = \sum_{k =1}^{n}{a_{{ik}}b_{{kj}}}$称为${AB}$的乘积，记为$C = AB$ 。

**4.** $\mathbf{A}^{\mathbf{T}}$**、**$\mathbf{A}^{\mathbf{-1}}$**、**$\mathbf{A}^{\mathbf{*}}$**三者之间的关系**

(1) ${(A^{T})}^{T} = A,{(AB)}^{T} = B^{T}A^{T},{(kA)}^{T} = kA^{T},{(A \pm B)}^{T} = A^{T} \pm B^{T}$

(2) $\left( A^{- 1} \right)^{- 1} = A,\left( {AB} \right)^{- 1} = B^{- 1}A^{- 1},\left( {kA} \right)^{- 1} = \frac{1}{k}A^{- 1},$

但 ${(A \pm B)}^{- 1} = A^{- 1} \pm B^{- 1}$不一定成立。

(3) $\left( A^{*} \right)^{*} = |A|^{n - 2}\ A\ \ (n \geq 3)$，$\left({AB} \right)^{*} = B^{*}A^{*},$ $\left( {kA} \right)^{*} = k^{n -1}A^{*}{\ \ }\left( n \geq 2 \right)$

但$\left( A \pm B \right)^{*} = A^{*} \pm B^{*}$不一定成立。

(4) ${(A^{- 1})}^{T} = {(A^{T})}^{- 1},\ \left( A^{- 1} \right)^{*} ={(AA^{*})}^{- 1},{(A^{*})}^{T} = \left( A^{T} \right)^{*}$

**5.有关**$\mathbf{A}^{\mathbf{*}}$**的结论**

(1) $AA^{*} = A^{*}A = |A|E$

(2) $|A^{*}| = |A|^{n - 1}\ (n \geq 2),\ \ \ \ {(kA)}^{*} = k^{n -1}A^{*},{{\ \ }\left( A^{*} \right)}^{*} = |A|^{n - 2}A(n \geq 3)$

(3) 若$A$可逆，则$A^{*} = |A|A^{- 1},{(A^{*})}^{*} = \frac{1}{|A|}A$

(4) 若$A$为$n$阶方阵，则：

$r(A^*)=\begin{cases}n,\quad r(A)=n\\ 1,\quad r(A)=n-1\\ 0,\quad r(A)<n-1\end{cases}$

**6.有关**$\mathbf{A}^{\mathbf{- 1}}$**的结论**

$A$可逆$\Leftrightarrow AB = E; \Leftrightarrow |A| \neq 0; \Leftrightarrow r(A) = n;$

$\Leftrightarrow A$可以表示为初等矩阵的乘积；$\Leftrightarrow A;\Leftrightarrow Ax = 0$。

**7.有关矩阵秩的结论**

(1) 秩$r(A)$=行秩=列秩；

(2) $r(A_{m \times n}) \leq \min(m,n);$

(3) $A \neq 0 \Rightarrow r(A) \geq 1$；

(4) $r(A \pm B) \leq r(A) + r(B);$

(5) 初等变换不改变矩阵的秩

(6) $r(A) + r(B) - n \leq r(AB) \leq \min(r(A),r(B)),$特别若$AB = O$
则：$r(A) + r(B) \leq n$

(7) 若$A^{- 1}$存在$\Rightarrow r(AB) = r(B);$ 若$B^{- 1}$存在
$\Rightarrow r(AB) = r(A);$

若$r(A_{m \times n}) = n \Rightarrow r(AB) = r(B);$ 若$r(A_{m \times s}) = n\Rightarrow r(AB) = r\left( A \right)$。

(8) $r(A_{m \times s}) = n \Leftrightarrow Ax = 0$只有零解

**8.分块求逆公式**

$\begin{pmatrix} A & O \\ O & B \\ \end{pmatrix}^{- 1} = \begin{pmatrix} A^{-1} & O \\ O & B^{- 1} \\ \end{pmatrix}$； $\begin{pmatrix} A & C \\ O & B \\\end{pmatrix}^{- 1} = \begin{pmatrix} A^{- 1}& - A^{- 1}CB^{- 1} \\ O & B^{- 1} \\ \end{pmatrix}$；

$\begin{pmatrix} A & O \\ C & B \\ \end{pmatrix}^{- 1} = \begin{pmatrix}  A^{- 1}&{O} \\   - B^{- 1}CA^{- 1} & B^{- 1} \\\end{pmatrix}$； $\begin{pmatrix} O & A \\ B & O \\ \end{pmatrix}^{- 1} =\begin{pmatrix} O & B^{- 1} \\ A^{- 1} & O \\ \end{pmatrix}$

这里$A$，$B$均为可逆方阵。

#### 向量

**1.有关向量组的线性表示**

(1)$\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$线性相关$\Leftrightarrow$至少有一个向量可以用其余向量线性表示。

(2)$\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$线性无关，$\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$，$\beta$线性相关$\Leftrightarrow \beta$可以由$\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$唯一线性表示。

(3) $\beta$可以由$\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$线性表示
$\Leftrightarrow r(\alpha_{1},\alpha_{2},\cdots,\alpha_{s}) =r(\alpha_{1},\alpha_{2},\cdots,\alpha_{s},\beta)$ 。

**2.有关向量组的线性相关性**

(1)部分相关，整体相关；整体无关，部分无关.

(2) ① $n$个$n$维向量
$\alpha_{1},\alpha_{2}\cdots\alpha_{n}$线性无关$\Leftrightarrow \left|\left\lbrack \alpha_{1}\alpha_{2}\cdots\alpha_{n} \right\rbrack \right| \neq0$， $n$个$n$维向量$\alpha_{1},\alpha_{2}\cdots\alpha_{n}$线性相关
$\Leftrightarrow |\lbrack\alpha_{1},\alpha_{2},\cdots,\alpha_{n}\rbrack| = 0$
。

② $n + 1$个$n$维向量线性相关。

③ 若$\alpha_{1},\alpha_{2}\cdots\alpha_{S}$线性无关，则添加分量后仍线性无关；或一组向量线性相关，去掉某些分量后仍线性相关。

**3.有关向量组的线性表示**

(1) $\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$线性相关$\Leftrightarrow$至少有一个向量可以用其余向量线性表示。

(2) $\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$线性无关，$\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$，$\beta$线性相关$\Leftrightarrow\beta$ 可以由$\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$唯一线性表示。

(3) $\beta$可以由$\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$线性表示
$\Leftrightarrow r(\alpha_{1},\alpha_{2},\cdots,\alpha_{s}) =r(\alpha_{1},\alpha_{2},\cdots,\alpha_{s},\beta)$

**4.向量组的秩与矩阵的秩之间的关系**

设$r(A_{m \times n}) =r$，则$A$的秩$r(A)$与$A$的行列向量组的线性相关性关系为：

(1) 若$r(A_{m \times n}) = r = m$，则$A$的行向量组线性无关。

(2) 若$r(A_{m \times n}) = r < m$，则$A$的行向量组线性相关。

(3) 若$r(A_{m \times n}) = r = n$，则$A$的列向量组线性无关。

(4) 若$r(A_{m \times n}) = r < n$，则$A$的列向量组线性相关。

**5.**$\mathbf{n}$**维向量空间的基变换公式及过渡矩阵**

若$\alpha_{1},\alpha_{2},\cdots,\alpha_{n}$与$\beta_{1},\beta_{2},\cdots,\beta_{n}$是向量空间$V$的两组基，则基变换公式为：

$(\beta_{1},\beta_{2},\cdots,\beta_{n}) = (\alpha_{1},\alpha_{2},\cdots,\alpha_{n})\begin{bmatrix}  c_{11}& c_{12}& \cdots & c_{1n} \\  c_{21}& c_{22}&\cdots & c_{2n} \\ \cdots & \cdots & \cdots & \cdots \\  c_{n1}& c_{n2} & \cdots & c_{{nn}} \\\end{bmatrix} = (\alpha_{1},\alpha_{2},\cdots,\alpha_{n})C$

其中$C$是可逆矩阵，称为由基$\alpha_{1},\alpha_{2},\cdots,\alpha_{n}$到基$\beta_{1},\beta_{2},\cdots,\beta_{n}$的过渡矩阵。

**6.坐标变换公式**

若向量$\gamma$在基$\alpha_{1},\alpha_{2},\cdots,\alpha_{n}$与基$\beta_{1},\beta_{2},\cdots,\beta_{n}$的坐标分别是
$X = {(x_{1},x_{2},\cdots,x_{n})}^{T}$，

$Y = \left( y_{1},y_{2},\cdots,y_{n} \right)^{T}$ 即： $\gamma =x_{1}\alpha_{1} + x_{2}\alpha_{2} + \cdots + x_{n}\alpha_{n} = y_{1}\beta_{1} +y_{2}\beta_{2} + \cdots + y_{n}\beta_{n}$，则向量坐标变换公式为$X = CY$ 或$Y = C^{- 1}X$，其中$C$是从基$\alpha_{1},\alpha_{2},\cdots,\alpha_{n}$到基$\beta_{1},\beta_{2},\cdots,\beta_{n}$的过渡矩阵。

**7.向量的内积**

$(\alpha,\beta) = a_{1}b_{1} + a_{2}b_{2} + \cdots + a_{n}b_{n} = \alpha^{T}\beta = \beta^{T}\alpha$

**8.Schmidt正交化**

若$\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$线性无关，则可构造$\beta_{1},\beta_{2},\cdots,\beta_{s}$使其两两正交，且$\beta_{i}$仅是$\alpha_{1},\alpha_{2},\cdots,\alpha_{i}$的线性组合$(i= 1,2,\cdots,n)$，再把$\beta_{i}$单位化，记$\gamma_{i} =\frac{\beta_{i}}{\left| \beta_{i}\right|}$，则$\gamma_{1},\gamma_{2},\cdots,\gamma_{i}$是规范正交向量组。其中
$\beta_{1} = \alpha_{1}$， $\beta_{2} = \alpha_{2} -\frac{(\alpha_{2},\beta_{1})}{(\beta_{1},\beta_{1})}\beta_{1}$ ， $\beta_{3} =\alpha_{3} - \frac{(\alpha_{3},\beta_{1})}{(\beta_{1},\beta_{1})}\beta_{1} -\frac{(\alpha_{3},\beta_{2})}{(\beta_{2},\beta_{2})}\beta_{2}$ ，

............

$\beta_{s} = \alpha_{s} - \frac{(\alpha_{s},\beta_{1})}{(\beta_{1},\beta_{1})}\beta_{1} - \frac{(\alpha_{s},\beta_{2})}{(\beta_{2},\beta_{2})}\beta_{2} - \cdots - \frac{(\alpha_{s},\beta_{s - 1})}{(\beta_{s - 1},\beta_{s - 1})}\beta_{s - 1}$

**9.正交基及规范正交基**

向量空间一组基中的向量如果两两正交，就称为正交基；若正交基中每个向量都是单位向量，就称其为规范正交基。

#### 线性方程组

**1．克莱姆法则**

线性方程组$\begin{cases}  a_{11}x_{1} + a_{12}x_{2} + \cdots +a_{1n}x_{n} = b_{1} \\   a_{21}x_{1} + a_{22}x_{2} + \cdots + a_{2n}x_{n} =b_{2} \\   \quad\cdots\cdots\cdots\cdots\cdots\cdots\cdots\cdots\cdots \\ a_{n1}x_{1} + a_{n2}x_{2} + \cdots + a_{{nn}}x_{n} = b_{n} \\ \end{cases}$，如果系数行列式$D = \left| A \right| \neq 0$，则方程组有唯一解，$x_{1} = \frac{D_{1}}{D},x_{2} = \frac{D_{2}}{D},\cdots,x_{n} =\frac{D_{n}}{D}$，其中$D_{j}$是把$D$中第$j$列元素换成方程组右端的常数列所得的行列式。

**2.** $n$阶矩阵$A$可逆$\Leftrightarrow Ax = 0$只有零解。$\Leftrightarrow\forall b,Ax = b$总有唯一解，一般地，$r(A_{m \times n}) = n \Leftrightarrow Ax= 0$只有零解。

**3.非奇次线性方程组有解的充分必要条件，线性方程组解的性质和解的结构**

(1) 设$A$为$m \times n$矩阵，若$r(A_{m \times n}) = m$，则对$Ax =b$而言必有$r(A) = r(A \vdots b) = m$，从而$Ax = b$有解。

(2) 设$x_{1},x_{2},\cdots x_{s}$为$Ax = b$的解，则$k_{1}x_{1} + k_{2}x_{2}\cdots + k_{s}x_{s}$当$k_{1} + k_{2} + \cdots + k_{s} = 1$时仍为$Ax =b$的解；但当$k_{1} + k_{2} + \cdots + k_{s} = 0$时，则为$Ax =0$的解。特别$\frac{x_{1} + x_{2}}{2}$为$Ax = b$的解；$2x_{3} - (x_{1} +x_{2})$为$Ax = 0$的解。

(3) 非齐次线性方程组${Ax} = b$无解$\Leftrightarrow r(A) + 1 =r(\overline{A}) \Leftrightarrow b$不能由$A$的列向量$\alpha_{1},\alpha_{2},\cdots,\alpha_{n}$线性表示。

**4.奇次线性方程组的基础解系和通解，解空间，非奇次线性方程组的通解**

(1) 齐次方程组${Ax} = 0$恒有解(必有零解)。当有非零解时，由于解向量的任意线性组合仍是该齐次方程组的解向量，因此${Ax}= 0$的全体解向量构成一个向量空间，称为该方程组的解空间，解空间的维数是$n - r(A)$，解空间的一组基称为齐次方程组的基础解系。

(2) $\eta_{1},\eta_{2},\cdots,\eta_{t}$是${Ax} = 0$的基础解系，即：

1) $\eta_{1},\eta_{2},\cdots,\eta_{t}$是${Ax} = 0$的解；

2) $\eta_{1},\eta_{2},\cdots,\eta_{t}$线性无关；

3) ${Ax} = 0$的任一解都可以由$\eta_{1},\eta_{2},\cdots,\eta_{t}$线性表出.
$k_{1}\eta_{1} + k_{2}\eta_{2} + \cdots + k_{t}\eta_{t}$是${Ax} = 0$的通解，其中$k_{1},k_{2},\cdots,k_{t}$是任意常数。

#### 矩阵的特征值和特征向量

**1.矩阵的特征值和特征向量的概念及性质**

(1) 设$\lambda$是$A$的一个特征值，则 ${kA},{aA} + {bE},A^{2},A^{m},f(A),A^{T},A^{- 1},A^{*}$有一个特征值分别为
${kλ},{aλ} + b,\lambda^{2},\lambda^{m},f(\lambda),\lambda,\lambda^{- 1},\frac{|A|}{\lambda},$且对应特征向量相同（$A^{T}$ 例外）。

(2)若$\lambda_{1},\lambda_{2},\cdots,\lambda_{n}$为$A$的$n$个特征值，则$\sum_{i= 1}^{n}\lambda_{i} = \sum_{i = 1}^{n}a_{{ii}},\prod_{i = 1}^{n}\lambda_{i}= |A|$ ,从而$|A| \neq 0 \Leftrightarrow A$没有特征值。

(3)设$\lambda_{1},\lambda_{2},\cdots,\lambda_{s}$为$A$的$s$个特征值，对应特征向量为$\alpha_{1},\alpha_{2},\cdots,\alpha_{s}$，

若: $\alpha = k_{1}\alpha_{1} + k_{2}\alpha_{2} + \cdots + k_{s}\alpha_{s}$ ,

则: $A^{n}\alpha = k_{1}A^{n}\alpha_{1} + k_{2}A^{n}\alpha_{2} + \cdots +k_{s}A^{n}\alpha_{s} = k_{1}\lambda_{1}^{n}\alpha_{1} +k_{2}\lambda_{2}^{n}\alpha_{2} + \cdots k_{s}\lambda_{s}^{n}\alpha_{s}$ 。

**2.相似变换、相似矩阵的概念及性质**

(1) 若$A \sim B$，则

1) $A^{T} \sim B^{T},A^{- 1} \sim B^{- 1},,A^{*} \sim B^{*}$

2) $|A| = |B|,\sum_{i = 1}^{n}A_{{ii}} = \sum_{i =1}^{n}b_{{ii}},r(A) = r(B)$

3) $|\lambda E - A| = |\lambda E - B|$，对$\forall\lambda$成立

**3.矩阵可相似对角化的充分必要条件**

(1)设$A$为$n$阶方阵，则$A$可对角化$\Leftrightarrow$对每个$k_{i}$重根特征值$\lambda_{i}$，有$n-r(\lambda_{i}E - A) = k_{i}$

(2) 设$A$可对角化，则由$P^{- 1}{AP} = \Lambda,$有$A = {PΛ}P^{-1}$，从而$A^{n} = P\Lambda^{n}P^{- 1}$

(3) 重要结论

1) 若$A \sim B,C \sim D$，则$\begin{bmatrix}  A & O \\ O & C \\\end{bmatrix} \sim \begin{bmatrix} B & O \\  O & D \\\end{bmatrix}$.

2) 若$A \sim B$，则$f(A) \sim f(B),\left| f(A) \right| \sim \left| f(B)\right|$，其中$f(A)$为关于$n$阶方阵$A$的多项式。

3) 若$A$为可对角化矩阵，则其非零特征值的个数(重根重复计算)＝秩($A$)

**4.实对称矩阵的特征值、特征向量及相似对角阵**

(1)相似矩阵：设$A,B$为两个$n$阶方阵，如果存在一个可逆矩阵$P$，使得$B =P^{- 1}{AP}$成立，则称矩阵$A$与$B$相似，记为$A \sim B$。

(2)相似矩阵的性质：如果$A \sim B$则有：

1) $A^{T} \sim B^{T}$

2) $A^{- 1} \sim B^{- 1}$ （若$A$，$B$均可逆）

3) $A^{k} \sim B^{k}$ （$k$为正整数）

4) $\left| {λE} - A \right| = \left| {λE} - B \right|$，从而$A,B$
有相同的特征值

5) $\left| A \right| = \left| B \right|$，从而$A,B$同时可逆或者不可逆

6) 秩$\left( A \right) =$秩$\left( B \right),\left| {λE} - A \right| =\left| {λE} - B \right|$，$A,B$不一定相似

#### 二次型

**1.**$\mathbf{n}$**个变量**$\mathbf{x}_{\mathbf{1}}\mathbf{,}\mathbf{x}_{\mathbf{2}}\mathbf{,\cdots,}\mathbf{x}_{\mathbf{n}}$**的二次齐次函数**

$f(x_{1},x_{2},\cdots,x_{n}) = \sum_{i = 1}^{n}{\sum_{j =1}^{n}{a_{{ij}}x_{i}y_{j}}}$，其中$a_{{ij}} = a_{{ji}}(i,j =1,2,\cdots,n)$，称为$n$元二次型，简称二次型. 若令$x = \ \begin{bmatrix}x_{1} \\ x_{1} \\  \vdots \\ x_{n} \\ \end{bmatrix},A = \begin{bmatrix}  a_{11}& a_{12}& \cdots & a_{1n} \\  a_{21}& a_{22}& \cdots & a_{2n} \\ \cdots &\cdots &\cdots &\cdots \\  a_{n1}& a_{n2} & \cdots & a_{{nn}} \\\end{bmatrix}$,这二次型$f$可改写成矩阵向量形式$f =x^{T}{Ax}$。其中$A$称为二次型矩阵，因为$a_{{ij}} =a_{{ji}}(i,j =1,2,\cdots,n)$，所以二次型矩阵均为对称矩阵，且二次型与对称矩阵一一对应，并把矩阵$A$的秩称为二次型的秩。

**2.惯性定理，二次型的标准形和规范形**

(1) 惯性定理

对于任一二次型，不论选取怎样的合同变换使它化为仅含平方项的标准型，其正负惯性指数与所选变换无关，这就是所谓的惯性定理。

(2) 标准形

二次型$f = \left( x_{1},x_{2},\cdots,x_{n} \right) =x^{T}{Ax}$经过合同变换$x = {Cy}$化为$f = x^{T}{Ax} =y^{T}C^{T}{AC}$

$y = \sum_{i = 1}^{r}{d_{i}y_{i}^{2}}$称为 $f(r \leq n)$的标准形。在一般的数域内，二次型的标准形不是唯一的，与所作的合同变换有关，但系数不为零的平方项的个数由$r(A)$唯一确定。

(3) 规范形

任一实二次型$f$都可经过合同变换化为规范形$f = z_{1}^{2} + z_{2}^{2} + \cdots z_{p}^{2} - z_{p + 1}^{2} - \cdots -z_{r}^{2}$，其中$r$为$A$的秩，$p$为正惯性指数，$r -p$为负惯性指数，且规范型唯一。

**3.用正交变换和配方法化二次型为标准形，二次型及其矩阵的正定性**

设$A$正定$\Rightarrow {kA}(k > 0),A^{T},A^{- 1},A^{*}$正定；$|A| >0$,$A$可逆；$a_{{ii}} > 0$，且$|A_{{ii}}| > 0$

$A$，$B$正定$\Rightarrow A +B$正定，但${AB}$，${BA}$不一定正定

$A$正定$\Leftrightarrow f(x) = x^{T}{Ax} > 0,\forall x \neq 0$

$\Leftrightarrow A$的各阶顺序主子式全大于零

$\Leftrightarrow A$的所有特征值大于零

$\Leftrightarrow A$的正惯性指数为$n$

$\Leftrightarrow$存在可逆阵$P$使$A = P^{T}P$

$\Leftrightarrow$存在正交矩阵$Q$，使$Q^{T}{AQ} = Q^{- 1}{AQ} =\begin{pmatrix} \lambda_{1} & & \\ \begin{matrix}  & \\  & \\ \end{matrix} &\ddots & \\  & & \lambda_{n} \\ \end{pmatrix},$

其中$\lambda_{i} > 0,i = 1,2,\cdots,n.$正定$\Rightarrow {kA}(k >0),A^{T},A^{- 1},A^{*}$正定； $|A| > 0,A$可逆；$a_{{ii}} >0$，且$|A_{{ii}}| > 0$ 。

### 概率论和数理统计

#### 随机事件和概率

**1.事件的关系与运算**

(1) 子事件：$A \subset B$，若$A$发生，则$B$发生。

(2) 相等事件：$A = B$，即$A \subset B$，且$B \subset A$ 。

(3) 和事件：$A\bigcup B$（或$A + B$），$A$与$B$中至少有一个发生。

(4) 差事件：$A - B$，$A$发生但$B$不发生。

(5) 积事件：$A\bigcap B$（或${AB}$），$A$与$B$同时发生。

(6) 互斥事件（互不相容）：$A\bigcap B$=$\varnothing$。

(7) 互逆事件（对立事件）：
$A\bigcap B=\varnothing ,A\bigcup B=\Omega ,A=\bar{B},B=\bar{A}$
**2.运算律**
(1) 交换律：$A\bigcup B=B\bigcup A,A\bigcap B=B\bigcap A$
(2) 结合律：$(A\bigcup B)\bigcup C=A\bigcup (B\bigcup C)$
(3) 分配律：$(A\bigcap B)\bigcap C=A\bigcap (B\bigcap C)$
**3.德$\centerdot $摩根律**

$\overline{A\bigcup B}=\bar{A}\bigcap \bar{B}$                 $\overline{A\bigcap B}=\bar{A}\bigcup \bar{B}$
**4.完全事件组** 

${{A}_{1}}{{A}_{2}}\cdots {{A}_{n}}$两两互斥，且和事件为必然事件，即${{A}_{i}}\bigcap {{A}_{j}}=\varnothing, i\ne j ,\underset{i=1}{\overset{n}{\mathop \bigcup }}\,=\Omega $

**5.概率的基本公式**
(1)条件概率:
 $P(B|A)=\frac{P(AB)}{P(A)}$,表示$A$发生的条件下，$B$发生的概率。
(2)全概率公式：
$P(A)=\sum\limits_{i=1}^{n}{P(A|{{B}_{i}})P({{B}_{i}}),{{B}_{i}}{{B}_{j}}}=\varnothing ,i\ne j,\underset{i=1}{\overset{n}{\mathop{\bigcup }}}\,{{B}_{i}}=\Omega $
(3) Bayes公式：

$P({{B}_{j}}|A)=\frac{P(A|{{B}_{j}})P({{B}_{j}})}{\sum\limits_{i=1}^{n}{P(A|{{B}_{i}})P({{B}_{i}})}},j=1,2,\cdots ,n$
注：上述公式中事件${{B}_{i}}$的个数可为可列个。
(4)乘法公式：
$P({{A}_{1}}{{A}_{2}})=P({{A}_{1}})P({{A}_{2}}|{{A}_{1}})=P({{A}_{2}})P({{A}_{1}}|{{A}_{2}})$
$P({{A}_{1}}{{A}_{2}}\cdots {{A}_{n}})=P({{A}_{1}})P({{A}_{2}}|{{A}_{1}})P({{A}_{3}}|{{A}_{1}}{{A}_{2}})\cdots P({{A}_{n}}|{{A}_{1}}{{A}_{2}}\cdots {{A}_{n-1}})$

**6.事件的独立性**
(1)$A$与$B$相互独立$\Leftrightarrow P(AB)=P(A)P(B)$
(2)$A$，$B$，$C$两两独立
$\Leftrightarrow P(AB)=P(A)P(B)$;$P(BC)=P(B)P(C)$ ;$P(AC)=P(A)P(C)$;
(3)$A$，$B$，$C$相互独立
$\Leftrightarrow P(AB)=P(A)P(B)$;     $P(BC)=P(B)P(C)$ ;
$P(AC)=P(A)P(C)$  ;   $P(ABC)=P(A)P(B)P(C)$

**7.独立重复试验** 

将某试验独立重复$n$次，若每次实验中事件A发生的概率为$p$，则$n$次试验中$A$发生$k$次的概率为：
$P(X=k)=C_{n}^{k}{{p}^{k}}{{(1-p)}^{n-k}}$
**8.重要公式与结论**
$(1)P(\bar{A})=1-P(A)$
$(2)P(A\bigcup B)=P(A)+P(B)-P(AB)$
   $P(A\bigcup B\bigcup C)=P(A)+P(B)+P(C)-P(AB)-P(BC)-P(AC)+P(ABC)$
$(3)P(A-B)=P(A)-P(AB)$
$(4)P(A\bar{B})=P(A)-P(AB),P(A)=P(AB)+P(A\bar{B}),$
 $P(A\bigcup B)=P(A)+P(\bar{A}B)=P(AB)+P(A\bar{B})+P(\bar{A}B)$
(5)条件概率$P(\centerdot |B)$满足概率的所有性质，
例如：. $P({{\bar{A}}_{1}}|B)=1-P({{A}_{1}}|B)$
$P({{A}_{1}}\bigcup {{A}_{2}}|B)=P({{A}_{1}}|B)+P({{A}_{2}}|B)-P({{A}_{1}}{{A}_{2}}|B)$
$P({{A}_{1}}{{A}_{2}}|B)=P({{A}_{1}}|B)P({{A}_{2}}|{{A}_{1}}B)$
(6)若${{A}_{1}},{{A}_{2}},\cdots ,{{A}_{n}}$相互独立，则$P(\bigcap\limits_{i=1}^{n}{{{A}_{i}}})=\prod\limits_{i=1}^{n}{P({{A}_{i}})},$
 $P(\bigcup\limits_{i=1}^{n}{{{A}_{i}}})=\prod\limits_{i=1}^{n}{(1-P({{A}_{i}}))}$
(7)互斥、互逆与独立性之间的关系：
$A$与$B$互逆$\Rightarrow$ $A$与$B$互斥，但反之不成立，$A$与$B$互斥（或互逆）且均非零概率事件$\Rightarrow $$A$与$B$不独立.
(8)若${{A}_{1}},{{A}_{2}},\cdots ,{{A}_{m}},{{B}_{1}},{{B}_{2}},\cdots ,{{B}_{n}}$相互独立，则$f({{A}_{1}},{{A}_{2}},\cdots ,{{A}_{m}})$与$g({{B}_{1}},{{B}_{2}},\cdots ,{{B}_{n}})$也相互独立，其中$f(\centerdot ),g(\centerdot )$分别表示对相应事件做任意事件运算后所得的事件，另外，概率为1（或0）的事件与任何事件相互独立.



#### 随机变量及其概率分布

**1.随机变量及概率分布**

取值带有随机性的变量，严格地说是定义在样本空间上，取值于实数的函数称为随机变量，概率分布通常指分布函数或分布律

**2.分布函数的概念与性质**

定义： $F(x) = P(X \leq x), - \infty < x < + \infty$

性质：(1)$0 \leq F(x) \leq 1$ 

(2) $F(x)$单调不减

(3) 右连续$F(x + 0) = F(x)$ 

(4) $F( - \infty) = 0,F( + \infty) = 1$

**3.离散型随机变量的概率分布**

$P(X = x_{i}) = p_{i},i = 1,2,\cdots,n,\cdots\quad\quad p_{i} \geq 0,\sum_{i =1}^{\infty}p_{i} = 1$

**4.连续型随机变量的概率密度**

概率密度$f(x)$;非负可积，且:

(1)$f(x) \geq 0,$ 

(2)$\int_{- \infty}^{+\infty}{f(x){dx} = 1}$ 

(3)$x$为$f(x)$的连续点，则:

$f(x) = F'(x)$分布函数$F(x) = \int_{- \infty}^{x}{f(t){dt}}$

**5.常见分布**

(1) 0-1分布:$P(X = k) = p^{k}{(1 - p)}^{1 - k},k = 0,1$

(2) 二项分布:$B(n,p)$： $P(X = k) = C_{n}^{k}p^{k}{(1 - p)}^{n - k},k =0,1,\cdots,n$

(3) **Poisson**分布:$p(\lambda)$： $P(X = k) = \frac{\lambda^{k}}{k!}e^{-\lambda},\lambda > 0,k = 0,1,2\cdots$

(4) 均匀分布$U(a,b)$：$f(x) = \{ \begin{matrix}  & \frac{1}{b - a},a < x< b \\  & 0, \\ \end{matrix} $

(5) 正态分布:$N(\mu,\sigma^{2}):$ $\varphi(x) =\frac{1}{\sqrt{2\pi}\sigma}e^{- \frac{{(x - \mu)}^{2}}{2\sigma^{2}}},\sigma > 0,\infty < x < + \infty$

(6)指数分布:$E(\lambda):f(x) =\{ \begin{matrix}  & \lambda e^{-{λx}},x > 0,\lambda > 0 \\  & 0, \\ \end{matrix} $

(7)几何分布:$G(p):P(X = k) = {(1 - p)}^{k - 1}p,0 < p < 1,k = 1,2,\cdots.$

(8)超几何分布: $H(N,M,n):P(X = k) = \frac{C_{M}^{k}C_{N - M}^{n -k}}{C_{N}^{n}},k =0,1,\cdots,min(n,M)$

**6.随机变量函数的概率分布**

(1)离散型：$P(X = x_{1}) = p_{i},Y = g(X)$

则: $P(Y = y_{j}) = \sum_{g(x_{i}) = y_{i}}^{}{P(X = x_{i})}$

(2)连续型：$X\tilde{\ }f_{X}(x),Y = g(x)$

则:$F_{y}(y) = P(Y \leq y) = P(g(X) \leq y) = \int_{g(x) \leq y}^{}{f_{x}(x)dx}$， $f_{Y}(y) = F'_{Y}(y)$

**7.重要公式与结论**

(1) $X\sim N(0,1) \Rightarrow \varphi(0) = \frac{1}{\sqrt{2\pi}},\Phi(0) =\frac{1}{2},$ $\Phi( - a) = P(X \leq - a) = 1 - \Phi(a)$

(2) $X\sim N\left( \mu,\sigma^{2} \right) \Rightarrow \frac{X -\mu}{\sigma}\sim N\left( 0,1 \right),P(X \leq a) = \Phi(\frac{a -\mu}{\sigma})$

(3) $X\sim E(\lambda) \Rightarrow P(X > s + t|X > s) = P(X > t)$

(4) $X\sim G(p) \Rightarrow P(X = m + k|X > m) = P(X = k)$

(5) 离散型随机变量的分布函数为阶梯间断函数；连续型随机变量的分布函数为连续函数，但不一定为处处可导函数。

(6) 存在既非离散也非连续型随机变量。

#### 多维随机变量及其分布

**1.二维随机变量及其联合分布**

由两个随机变量构成的随机向量$(X,Y)$， 联合分布为$F(x,y) = P(X \leq x,Y \leq y)$

**2.二维离散型随机变量的分布**

(1) 联合概率分布律 $P\{ X = x_{i},Y = y_{j}\} = p_{{ij}};i,j =1,2,\cdots$

(2) 边缘分布律 $p_{i \cdot} = \sum_{j = 1}^{\infty}p_{{ij}},i =1,2,\cdots$ $p_{\cdot j} = \sum_{i}^{\infty}p_{{ij}},j = 1,2,\cdots$

(3) 条件分布律 $P\{ X = x_{i}|Y = y_{j}\} = \frac{p_{{ij}}}{p_{\cdot j}}$
$P\{ Y = y_{j}|X = x_{i}\} = \frac{p_{{ij}}}{p_{i \cdot}}$

**3. 二维连续性随机变量的密度**

(1) 联合概率密度$f(x,y):$

1) $f(x,y) \geq 0$ 

2) $\int_{- \infty}^{+ \infty}{\int_{- \infty}^{+ \infty}{f(x,y)dxdy}} = 1$

(2) 分布函数：$F(x,y) = \int_{- \infty}^{x}{\int_{- \infty}^{y}{f(u,v)dudv}}$

(3) 边缘概率密度： $f_{X}\left( x \right) = \int_{- \infty}^{+ \infty}{f\left( x,y \right){dy}}$ $f_{Y}(y) = \int_{- \infty}^{+ \infty}{f(x,y)dx}$

(4) 条件概率密度：$f_{X|Y}\left( x \middle| y \right) = \frac{f\left( x,y \right)}{f_{Y}\left( y \right)}$ $f_{Y|X}(y|x) = \frac{f(x,y)}{f_{X}(x)}$

**4.常见二维随机变量的联合分布**

(1) 二维均匀分布：$(x,y) \sim U(D)$ ,$f(x,y) = \begin{cases} \frac{1}{S(D)},(x,y) \in D \\   0,其他  \end{cases}$

(2) 二维正态分布：$(X,Y)\sim N(\mu_{1},\mu_{2},\sigma_{1}^{2},\sigma_{2}^{2},\rho)$,$(X,Y)\sim N(\mu_{1},\mu_{2},\sigma_{1}^{2},\sigma_{2}^{2},\rho)$

$f(x,y) = \frac{1}{2\pi\sigma_{1}\sigma_{2}\sqrt{1 - \rho^{2}}}.\exp\left\{ \frac{- 1}{2(1 - \rho^{2})}\lbrack\frac{{(x - \mu_{1})}^{2}}{\sigma_{1}^{2}} - 2\rho\frac{(x - \mu_{1})(y - \mu_{2})}{\sigma_{1}\sigma_{2}} + \frac{{(y - \mu_{2})}^{2}}{\sigma_{2}^{2}}\rbrack \right\}$

**5.随机变量的独立性和相关性**

$X$和$Y$的相互独立:$\Leftrightarrow F\left( x,y \right) = F_{X}\left( x \right)F_{Y}\left( y \right)$:

$\Leftrightarrow p_{{ij}} = p_{i \cdot} \cdot p_{\cdot j}$（离散型）
$\Leftrightarrow f\left( x,y \right) = f_{X}\left( x \right)f_{Y}\left( y \right)$（连续型）

$X$和$Y$的相关性：

相关系数$\rho_{{XY}} = 0$时，称$X$和$Y$不相关，
否则称$X$和$Y$相关

**6.两个随机变量简单函数的概率分布**

离散型： $P\left( X = x_{i},Y = y_{i} \right) = p_{{ij}},Z = g\left( X,Y \right)$ 则：

$P(Z = z_{k}) = P\left\{ g\left( X,Y \right) = z_{k} \right\} = \sum_{g\left( x_{i},y_{i} \right) = z_{k}}^{}{P\left( X = x_{i},Y = y_{j} \right)}$

连续型： $\left( X,Y \right) \sim f\left( x,y \right),Z = g\left( X,Y \right)$
则：

$F_{z}\left( z \right) = P\left\{ g\left( X,Y \right) \leq z \right\} = \iint_{g(x,y) \leq z}^{}{f(x,y)dxdy}$，$f_{z}(z) = F'_{z}(z)$

**7.重要公式与结论**

(1) 边缘密度公式： $f_{X}(x) = \int_{- \infty}^{+ \infty}{f(x,y)dy,}$
$f_{Y}(y) = \int_{- \infty}^{+ \infty}{f(x,y)dx}$

(2) $P\left\{ \left( X,Y \right) \in D \right\} = \iint_{D}^{}{f\left( x,y \right){dxdy}}$

(3) 若$(X,Y)$服从二维正态分布$N(\mu_{1},\mu_{2},\sigma_{1}^{2},\sigma_{2}^{2},\rho)$
则有：

1) $X\sim N\left( \mu_{1},\sigma_{1}^{2} \right),Y\sim N(\mu_{2},\sigma_{2}^{2}).$

2) $X$与$Y$相互独立$\Leftrightarrow \rho = 0$，即$X$与$Y$不相关。

3) $C_{1}X + C_{2}Y\sim N(C_{1}\mu_{1} + C_{2}\mu_{2},C_{1}^{2}\sigma_{1}^{2} + C_{2}^{2}\sigma_{2}^{2} + 2C_{1}C_{2}\sigma_{1}\sigma_{2}\rho)$

4) ${\ X}$关于$Y=y$的条件分布为： $N(\mu_{1} + \rho\frac{\sigma_{1}}{\sigma_{2}}(y - \mu_{2}),\sigma_{1}^{2}(1 - \rho^{2}))$

5) $Y$关于$X = x$的条件分布为： $N(\mu_{2} + \rho\frac{\sigma_{2}}{\sigma_{1}}(x - \mu_{1}),\sigma_{2}^{2}(1 - \rho^{2}))$

(4) 若$X$与$Y$独立，且分别服从$N(\mu_{1},\sigma_{1}^{2}),N(\mu_{1},\sigma_{2}^{2}),$
则：$\left( X,Y \right)\sim N(\mu_{1},\mu_{2},\sigma_{1}^{2},\sigma_{2}^{2},0),$

$C_{1}X + C_{2}Y\tilde{\ }N(C_{1}\mu_{1} + C_{2}\mu_{2},C_{1}^{2}\sigma_{1}^{2} C_{2}^{2}\sigma_{2}^{2}).$

(5) 若$X$与$Y$相互独立，$f\left( x \right)$和$g\left( x \right)$为连续函数， 则$f\left( X \right)$和$g(Y)$也相互独立。

#### 随机变量的数字特征

**1.数学期望**

离散型：$P\left\{ X = x_{i} \right\} = p_{i},E(X) = \sum_{i}^{}{x_{i}p_{i}}$；

连续型： $X\sim f(x),E(X) = \int_{- \infty}^{+ \infty}{xf(x)dx}$

性质：

(1) $E(C) = C,E\lbrack E(X)\rbrack = E(X)$

(2) $E(C_{1}X + C_{2}Y) = C_{1}E(X) + C_{2}E(Y)$

(3) 若$X$和$Y$独立，则$E(XY) = E(X)E(Y)$ 

(4)$\left\lbrack E(XY) \right\rbrack^{2} \leq E(X^{2})E(Y^{2})$

**2.方差**：$D(X) = E\left\lbrack X - E(X) \right\rbrack^{2} = E(X^{2}) - \left\lbrack E(X) \right\rbrack^{2}$

**3.标准差**：$\sqrt{D(X)}$，

**4.离散型：**$D(X) = \sum_{i}^{}{\left\lbrack x_{i} - E(X) \right\rbrack^{2}p_{i}}$

**5.连续型：**$D(X) = {\int_{- \infty}^{+ \infty}\left\lbrack x - E(X) \right\rbrack}^{2}f(x)dx$

性质：

(1)$\ D(C) = 0,D\lbrack E(X)\rbrack = 0,D\lbrack D(X)\rbrack = 0$

(2) $X$与$Y$相互独立，则$D(X \pm Y) = D(X) + D(Y)$

(3)$\ D\left( C_{1}X + C_{2} \right) = C_{1}^{2}D\left( X \right)$

(4) 一般有 $D(X \pm Y) = D(X) + D(Y) \pm 2Cov(X,Y) = D(X) + D(Y) \pm 2\rho\sqrt{D(X)}\sqrt{D(Y)}$

(5)$\ D\left( X \right) < E\left( X - C \right)^{2},C \neq E\left( X \right)$

(6)$\ D(X) = 0 \Leftrightarrow P\left\{ X = C \right\} = 1$

**6.随机变量函数的数学期望**

(1) 对于函数$Y = g(x)$

$X$为离散型：$P\{ X = x_{i}\} = p_{i},E(Y) = \sum_{i}^{}{g(x_{i})p_{i}}$；

$X$为连续型：$X\sim f(x),E(Y) = \int_{- \infty}^{+ \infty}{g(x)f(x)dx}$

(2) $Z = g(X,Y)$;$\left( X,Y \right)\sim P\{ X = x_{i},Y = y_{j}\} = p_{{ij}}$; $E(Z) = \sum_{i}^{}{\sum_{j}^{}{g(x_{i},y_{j})p_{{ij}}}}$ $\left( X,Y \right)\sim f(x,y)$;$E(Z) = \int_{- \infty}^{+ \infty}{\int_{- \infty}^{+ \infty}{g(x,y)f(x,y)dxdy}}$

**7.协方差** 

$Cov(X,Y) = E\left\lbrack (X - E(X)(Y - E(Y)) \right\rbrack$

**8.相关系数**

 $\rho_{{XY}} = \frac{Cov(X,Y)}{\sqrt{D(X)}\sqrt{D(Y)}}$,$k$阶原点矩 $E(X^{k})$;
$k$阶中心矩 $E\left\{ {\lbrack X - E(X)\rbrack}^{k} \right\}$

性质：

(1)$\ Cov(X,Y) = Cov(Y,X)$

(2)$\ Cov(aX,bY) = abCov(Y,X)$

(3)$\ Cov(X_{1} + X_{2},Y) = Cov(X_{1},Y) + Cov(X_{2},Y)$

(4)$\ \left| \rho\left( X,Y \right) \right| \leq 1$

(5) $\ \rho\left( X,Y \right) = 1 \Leftrightarrow P\left( Y = aX + b \right) = 1$ ，其中$a > 0$

$\rho\left( X,Y \right) = - 1 \Leftrightarrow P\left( Y = aX + b \right) = 1$
，其中$a < 0$

**9.重要公式与结论**

(1)$\ D(X) = E(X^{2}) - E^{2}(X)$

(2)$\ Cov(X,Y) = E(XY) - E(X)E(Y)$

(3) $\left| \rho\left( X,Y \right) \right| \leq 1,$且 $\rho\left( X,Y \right) = 1 \Leftrightarrow P\left( Y = aX + b \right) = 1$，其中$a > 0$

$\rho\left( X,Y \right) = - 1 \Leftrightarrow P\left( Y = aX + b \right) = 1$，其中$a < 0$

(4) 下面5个条件互为充要条件：

$\rho(X,Y) = 0$ $\Leftrightarrow Cov(X,Y) = 0$ $\Leftrightarrow E(X,Y) = E(X)E(Y)$ $\Leftrightarrow D(X + Y) = D(X) + D(Y)$ $\Leftrightarrow  D(X - Y) = D(X) + D(Y)$

注：$X$与$Y$独立为上述5个条件中任何一个成立的充分条件，但非必要条件。

#### 数理统计的基本概念

**1.基本概念**

总体：研究对象的全体，它是一个随机变量，用$X$表示。

个体：组成总体的每个基本元素。

简单随机样本：来自总体$X$的$n$个相互独立且与总体同分布的随机变量$X_{1},X_{2}\cdots,X_{n}$，称为容量为$n$的简单随机样本，简称样本。

统计量：设$X_{1},X_{2}\cdots,X_{n},$是来自总体$X$的一个样本，$g(X_{1},X_{2}\cdots,X_{n})$）是样本的连续函数，且$g()$中不含任何未知参数，则称$g(X_{1},X_{2}\cdots,X_{n})$为统计量。

样本均值：$\overline{X} = \frac{1}{n}\sum_{i = 1}^{n}X_{i}$

样本方差：$S^{2} = \frac{1}{n - 1}\sum_{i = 1}^{n}{(X_{i} - \overline{X})}^{2}$

样本矩：样本$k$阶原点矩：$A_{k} = \frac{1}{n}\sum_{i = 1}^{n}X_{i}^{k},k = 1,2,\cdots$

样本$k$阶中心矩：$B_{k} = \frac{1}{n}\sum_{i = 1}^{n}{(X_{i} - \overline{X})}^{k},k = 1,2,\cdots$

**2.分布**

$\chi^{2}$分布：$\chi^{2} = X_{1}^{2} + X_{2}^{2} + \cdots + X_{n}^{2}\sim\chi^{2}(n)$，其中$X_{1},X_{2}\cdots,X_{n},$相互独立，且同服从$N(0,1)$

$t$分布：$T = \frac{X}{\sqrt{Y/n}}\sim t(n)$ ，其中$X\sim N\left( 0,1 \right),Y\sim\chi^{2}(n),$且$X$，$Y$ 相互独立。

$F$分布：$F = \frac{X/n_{1}}{Y/n_{2}}\sim F(n_{1},n_{2})$，其中$X\sim\chi^{2}\left( n_{1} \right),Y\sim\chi^{2}(n_{2}),$且$X$，$Y$相互独立。

分位数：若$P(X \leq x_{\alpha}) = \alpha,$则称$x_{\alpha}$为$X$的$\alpha$分位数

**3.正态总体的常用样本分布**

(1) 设$X_{1},X_{2}\cdots,X_{n}$为来自正态总体$N(\mu,\sigma^{2})$的样本，

$\overline{X} = \frac{1}{n}\sum_{i = 1}^{n}X_{i},S^{2} = \frac{1}{n - 1}\sum_{i = 1}^{n}{{(X_{i} - \overline{X})}^{2},}$则：

1) $\overline{X}\sim N\left( \mu,\frac{\sigma^{2}}{n} \right){\ \ }$或者$\frac{\overline{X} - \mu}{\frac{\sigma}{\sqrt{n}}}\sim N(0,1)$

2) $\frac{(n - 1)S^{2}}{\sigma^{2}} = \frac{1}{\sigma^{2}}\sum_{i = 1}^{n}{{(X_{i} - \overline{X})}^{2}\sim\chi^{2}(n - 1)}$

3) $\frac{1}{\sigma^{2}}\sum_{i = 1}^{n}{{(X_{i} - \mu)}^{2}\sim\chi^{2}(n)}$

4)${\ \ }\frac{\overline{X} - \mu}{S/\sqrt{n}}\sim t(n - 1)$

**4.重要公式与结论**

(1) 对于$\chi^{2}\sim\chi^{2}(n)$，有$E(\chi^{2}(n)) = n,D(\chi^{2}(n)) = 2n;$

(2) 对于$T\sim t(n)$，有$E(T) = 0,D(T) = \frac{n}{n - 2}(n > 2)$；

(3) 对于$F\tilde{\ }F(m,n)$，有 $\frac{1}{F}\sim F(n,m),F_{a/2}(m,n) = \frac{1}{F_{1 - a/2}(n,m)};$

(4) 对于任意总体$X$，有 $E(\overline{X}) = E(X),E(S^{2}) = D(X),D(\overline{X}) = \frac{D(X)}{n}$



================================================
FILE: 0.math/1.CS229/README.md
================================================
# CS 229 机器学习课程复习材料

文件夹的材料斯坦福大学CS 229机器学习课程的基础材料的翻译版本，[原始文件下载](http://cs229.stanford.edu/summer2019/cs229-linalg.pdf)

翻译：[黄海广](https://github.com/fengdu78)
备注：请关注[github](https://github.com/fengdu78/Data-Science-Notes/tree/master/0.math)的更新，近期将更新完。

markdown文件夹是文件的markdown代码，内容与pdf一致，是为了方便研究者写论文或者文章使用。

github内容是方便用户学习研究使用，请勿用于商业目的。

如果需要引用这个Repo:

格式： `fengdu78, Data-Science-Notes, (2019), GitHub repository, https://github.com/fengdu78/Data-Science-Notes`

================================================
FILE: 0.math/1.CS229/markdown/1.CS229-LinearAlgebra.md
================================================
> 本文是斯坦福大学CS 229机器学习课程的基础材料，[原始文件下载](http://cs229.stanford.edu/summer2019/cs229-linalg.pdf)

> 原文作者：Zico Kolter，修改：Chuong Do， Tengyu Ma
>
> 翻译：[黄海广](https://github.com/fengdu78)
> 备注：请关注[github](https://github.com/fengdu78/Data-Science-Notes/tree/master/0.math)的更新，线性代数和概率论已经更新完毕。



# CS229 机器学习课程复习材料-线性代数

[TOC]
## 线性代数复习和参考

### 1.  基础概念和符号

线性代数提供了一种紧凑地表示和操作线性方程组的方法。 例如，以下方程组：
$$
4x_1 − 5x_2 = −13
$$
$$
−2x_1 + 3x_2 = 9
$$

这是两个方程和两个变量，正如你从高中代数中所知，你可以找到 $x_1$ 和 $x_2$ 的唯一解（除非方程以某种方式退化，例如，如果第二个方程只是第一个的倍数，但在上面的情况下，实际上只有一个唯一解）。 在矩阵表示法中，我们可以更紧凑地表达：

$$
Ax= b
$$

$$
\text { with } A=\left[\begin{array}{cc}{4} & {-5} \\ {-2} & {3}\end{array}\right], b=\left[\begin{array}{c}{-13} \\ {9}\end{array}\right]
$$

我们可以看到，这种形式的线性方程有许多优点（比如明显地节省空间）。

#### 1.1 基本符号

我们使用以下符号：

-   $A \in \mathbb{R}^{m \times n}$，表示 $A$ 为由实数组成具有$m$行和$n$列的矩阵。

-   $x \in \mathbb{R}^{ n}$，表示具有$n$个元素的向量。 通常，向量$x$将表示列向量: 即，具有$n$行和$1$列的矩阵。 如果我们想要明确地表示行向量: 具有 $1$ 行和$n$列的矩阵 - 我们通常写$x^T$（这里$x^T$$x$的转置）。

-   $x_i$表示向量$x$的第$i$个元素

$$
x=\left[\begin{array}{c}{x_{1}} \\ {x_{2}} \\ {\vdots} \\ {x_{n}}\end{array}\right]
$$

-   我们使用符号 $a_{ij}$（或$A_{ij}$,$A_{i,j}$等）来表示第 $i$ 行和第$j$列中的 $A$ 的元素：

$$
A=\left[\begin{array}{cccc}{a_{11}} & {a_{12}} & {\cdots} & {a_{1 n}} \\ {a_{21}} & {a_{22}} & {\cdots} & {a_{2 n}} \\ {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ {a_{m 1}} & {a_{m 2}} & {\cdots} & {a_{m n}}\end{array}\right]
$$

-   我们用$a^j$或者$A_{:,j}$表示矩阵$A$的第$j$列：

$$
A=\left[\begin{array}{llll}{ |} & { |} & {} & { |} \\ {a^{1}} & {a^{2}} & {\cdots} & {a^{n}} \\ { |} & { |} & {} & { |}\end{array}\right]
$$

-   我们用$a^T_i$或者$A_{i,:}$表示矩阵$A$的第$i$行：
$$
A=\left[\begin{array}{c}{-a_{1}^{T}-} \\ {-a_{2}^{T}-} \\ {\vdots} \\ {-a_{m}^{T}-}\end{array}\right]
$$


-   在许多情况下，将矩阵视为列向量或行向量的集合非常重要且方便。 通常，在向量而不是标量上操作在数学上（和概念上）更清晰。只要明确定义了符号，用于矩阵的列或行的表示方式并没有通用约定。

### 2.矩阵乘法

两个矩阵相乘，其中 $A \in \mathbb{R}^{m \times n}$  and $B \in \mathbb{R}^{n \times p}$ ，则：

$$
C = AB \in \mathbb{R}^{m \times p}
$$


其中：

$$
C_{i j}=\sum_{k=1}^{n} A_{i k} B_{k j}
$$

请注意，为了使矩阵乘积存在，$A$中的列数必须等于$B$中的行数。有很多方法可以查看矩阵乘法，我们将从检查一些特殊情况开始。

#### 2.1 向量-向量乘法

给定两个向量$x, y \in \mathbb{R}^{n}$,$x^T y$通常称为**向量内积**或者**点积**，结果是个**实数**。

$$
x^{T} y \in \mathbb{R}=\left[\begin{array}{llll}{x_{1}} & {x_{2}} & {\cdots} & {x_{n}}\end{array}\right]\left[\begin{array}{c}{y_{1}} \\ {y_{2}} \\ {\vdots} \\ {y_{n}}\end{array}\right]=\sum_{i=1}^{n} x_{i} y_{i}
$$

注意：$x^T y = y^Tx$ 始终成立。

给定向量 $x \in \mathbb{R}^{m}$, $y \in \mathbb{R}^{n}$ (他们的维度是否相同都没关系)，$xy^T \in \mathbb{R}^{m \times n}$叫做**向量外积 ** , 当 $(xy^T)_{ij} = x_iy_j$ 的时候，它是一个矩阵。

$$
x y^{T} \in \mathbb{R}^{m \times n}=\left[\begin{array}{c}{x_{1}} \\ {x_{2}} \\ {\vdots} \\ {x_{m}}\end{array}\right]\left[\begin{array}{llll}{y_{1}} & {y_{2}} & {\cdots} & {y_{n}}\end{array}\right]=\left[\begin{array}{cccc}{x_{1} y_{1}} & {x_{1} y_{2}} & {\cdots} & {x_{1} y_{n}} \\ {x_{2} y_{1}} & {x_{2} y_{2}} & {\cdots} & {x_{2} y_{n}} \\ {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ {x_{m} y_{1}} & {x_{m} y_{2}} & {\cdots} & {x_{m} y_{n}}\end{array}\right]
$$
举一个外积如何使用的一个例子：让$1\in R^{n}$表示一个$n$维向量，其元素都等于1，此外，考虑矩阵$A \in R^{m \times n}$，其列全部等于某个向量 $x \in R^{m}$。 我们可以使用外积紧凑地表示矩阵 $A$:

$$
A=\left[\begin{array}{llll}{ |} & { |} & {} & { |} \\ {x} & {x} & {\cdots} & {x} \\ { |} & { |} & {} & { |}\end{array}\right]=\left[\begin{array}{cccc}{x_{1}} & {x_{1}} & {\cdots} & {x_{1}} \\ {x_{2}} & {x_{2}} & {\cdots} & {x_{2}} \\ {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ {x_{m}} & {x_{m}} & {\cdots} & {x_{m}}\end{array}\right]=\left[\begin{array}{c}{x_{1}} \\ {x_{2}} \\ {\vdots} \\ {x_{m}}\end{array}\right]\left[\begin{array}{lll}{1} & {1} & {\cdots} & {1}\end{array}\right]=x \mathbf{1}^{T}
$$
#### 2.2 矩阵-向量乘法

给定矩阵 $A \in \mathbb{R}^{m \times n}$，向量 $x \in  \mathbb{R}^{n}$ , 它们的积是一个向量 $y = Ax \in R^{m}$。 有几种方法可以查看矩阵向量乘法，我们将依次查看它们中的每一种。

如果我们按行写$A$，那么我们可以表示$Ax$为：

$$
y=A x=\left[\begin{array}{ccc}{-} & {a_{1}^{T}} & {-} \\ {-} & {a_{2}^{T}} & {-} \\ {} & {\vdots} & {} \\ {-} & {a_{m}^{T}} & {-}\end{array}\right] x=\left[\begin{array}{c}{a_{1}^{T} x} \\ {a_{2}^{T} x} \\ {\vdots} \\ {a_{m}^{T} x}\end{array}\right]
$$

换句话说，第$i$个$y$是$A$的第$i$行和$x$的内积，即：$y_i = y_{i}=a_{i}^{T} x$。

同样的， 可以把 $A$ 写成列的方式，则公式如下：

$$
y=A x=\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {a^{1}} & {a^{2}} & {\cdots} & {a^{n}} \\ { |} & { |} & {} & { |}\end{array}\right]\left[\begin{array}{c}{x_{1}} \\ {x_{2}} \\ {\vdots} \\ {x_{n}}\end{array}\right]=\left[\begin{array}{c}{ } \\ {a^{1}{ } \\ }\end{array}\right] x_{1}+\left[\begin{array}{c}{ } \\ {a^{2}{ } \\ }\end{array}\right] x_{2}+{\cdots} +\left[\begin{array}{c}{ } \\ {a^{n}{ } \\ }\end{array}\right] x_{n}
$$

换句话说，$y$是$A$的列的线性组合，其中线性组合的系数由$x$的元素给出。

到目前为止，我们一直在右侧乘以列向量，但也可以在左侧乘以行向量。 这是写的，$y^T = x^TA$ 表示$A \in \mathbb{R}^{m \times n}$，$x \in \mathbb{R}^{m}$，$y \in \mathbb{R}^{n}$。 和以前一样，我们可以用两种可行的方式表达$y^T$，这取决于我们是否根据行或列表达$A$.

第一种情况，我们把$A$用列表示：

$$
y^{T}=x^{T} A=x^{T}\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {a^{1}} & {a^{2}} & {\cdots} & {a^{n}} \\ { |} & { |} & {} & { |}\end{array}\right]=\left[\begin{array}{cccc}{x^{T} a^{1}} & {x^{T} a^{2}} & {\dots} & {x^{T} a^{n}}\end{array}\right]
$$

这表明$y^T$的第$i$个元素等于$x$和$A$的第$i$列的内积。

最后，根据行表示$A$，我们得到了向量-矩阵乘积的最终表示:

$$
y^T=x^TA
=\left[\begin{array}{llll}{x_{1}} & {x_{2}} & {\cdots} & {x_{n}}\end{array}\right]\left[\begin{array}{c}{-a_{1}^{T}-} \\ {-a_{2}^{T}-} \\ {\vdots} \\ {-a_{m}^{T}-}\end{array}\right]

=x_{1}\left[-a_{1}^{T}-\right]+x_{2}\left[-a_{2}^{T}-\right]+\ldots+x_{n}\left[-a_{n}^{T}-\right]
$$
所以我们看到$y^T$是$A$的行的线性组合，其中线性组合的系数由$x$的元素给出。

#### 2.3 矩阵-矩阵乘法

有了这些知识，我们现在可以看看四种不同的（形式不同，但结果是相同的）矩阵-矩阵乘法：也就是本节开头所定义的$C=AB$的乘法。

首先，我们可以将矩阵 - 矩阵乘法视为一组向量-向量乘积。 从定义中可以得出：最明显的观点是$C $的$( i，j )$元素等于$A$的第$i$行和$B$的的$j$列的内积。如下面的公式所示：
$$
C=A B=\left[\begin{array}{cc}{-} & {a_{1}^{T}} &{-} \\ {-} & {a_{2}^{T}} &{-}  \\ {} & {\vdots} \\ {-} & {a_{m}^{T}} &{-} \end{array}\right]\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {b_{1}} & {b_{2}} & {\cdots} & {b_{p}} \\ { |} & { |} & {} & { |}\end{array}\right]=\left[\begin{array}{cccc}{a_{1}^{T} b_{1}} & {a_{1}^{T} b_{2}} & {\cdots} & {a_{1}^{T} b_{p}} \\ {a_{2}^{T} b_{1}} & {a_{2}^{T} b_{2}} & {\cdots} & {a_{2}^{T} b_{p}} \\ {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ {a_{m}^{T} b_{1}} & {a_{m}^{T} b_{2}} & {\cdots} & {a_{m}^{T} b_{p}}\end{array}\right]
$$

这里的$ A \in \mathbb{R}^{m\times n}$ ，$B \in \mathbb{R}^{n \times p}$， $a_i \in \mathbb{R}^n$ ，$b^j \in \mathbb{R}^{n \times p}$， 这里的$  A \in \mathbb{R}^ {m \times n}，$ $B \in \mathbb{R}^ {n \times p} $， $a_i \in \mathbb{R} ^ n $，$ b ^ j \in \mathbb{R} ^ {n \times p} $，所以它们可以计算内积。 我们用通常用行表示$ A $而用列表示$B$。
或者，我们可以用列表示$ A$，用行表示$B $，这时$AB$是求外积的和。公式如下：
$$
C=A B=\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {a_{1}} & {a_{2}} & {\cdots} & {a_{n}} \\ { |} & { |} & {} & { |}\end{array}\right]\left[\begin{array}{c}{-}& {b_{1}^{T}}&{-} \\ {-}& {b_{2}^{T}}&{-}  \\ {\vdots} \\{-}& {b_{n}^{T}}&{-}\end{array}\right]=\sum_{i=1}^{n} a_{i} b_{i}^{T}
$$
换句话说，$AB$等于所有的$A$的第$i$列和$B$第$i$行的外积的和。因此，在这种情况下， $a_i \in \mathbb{R}^ m $和$b_i \in \mathbb{R}^p$， 外积$a^ib_i^T$的维度是$m×p$，与$C$的维度一致。

其次，我们还可以将矩阵 - 矩阵乘法视为一组矩阵向量积。如果我们把$B$用列表示，我们可以将$C$的列视为$A$和$B$的列的矩阵向量积。公式如下：

$$
C=A B=A\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {b_{1}} & {b_{2}} & {\cdots} & {b_{p}} \\ { |} & { |} & {} & { |}\end{array}\right]=\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {A b_{1}} & {A b_{2}} & {\cdots} & {A b_{p}} \\ { |} & { |} & {} & { |}\end{array}\right]
$$
这里$C$的第$i$列由矩阵向量乘积给出，右边的向量为$c_i = Ab_i$。 这些矩阵向量乘积可以使用前一小节中给出的两个观点来解释。
最后，我们有类似的观点，我们用行表示$A$，$C$的行作为$A$和$C$行之间的矩阵向量积。公式如下：
$$
C=A B=\left[\begin{array}{ccc}{-} & {a_{1}^{T}} & {-} \\ {-} & {a_{2}^{T}} & {-} \\ {} & {\vdots} & {} \\ {-} & {a_{m}^{T}} & {-}\end{array}\right]  B=\left[\begin{array}{c} {-} & {a_{1}^{T} B} & {-}\\ {-} & {a_{2}^{T} B} & {-} \\ {\vdots} \\ {-} & {a_{m}^{T} B}& {-}\end{array}\right]
$$

这里第$i$行的$C$由左边的向量的矩阵向量乘积给出：$c_i^T = a_i^T B$

将矩阵乘法剖析到如此大的程度似乎有点过分，特别是当所有这些观点都紧跟在我们在本节开头给出的初始定义（在一行数学中）之后。 

这些不同方法的直接优势在于它们允许您**在向量的级别/单位而不是标量上进行操作**。 为了完全理解线性代数而不会迷失在复杂的索引操作中，关键是要用尽可能多的概念进行操作。

实际上所有的线性代数都处理某种矩阵乘法，花一些时间对这里提出的观点进行直观的理解是非常必要的。 

除此之外，了解一些更高级别的矩阵乘法的基本属性是很有必要的：

-   矩阵乘法结合律: $(AB)C = A(BC)$

-   矩阵乘法分配律: $A(B + C) = AB + AC$

-   矩阵乘法通常不是可交换的; 也就是说，通常$AB \ne BA$。 （例如，假设$  A \in \mathbb{R}^ {m \times n}，$ $B \in \mathbb{R}^ {n \times p} $，如果$m$和$q$不相等，矩阵乘积$BA$甚至不存在！）

如果您不熟悉这些属性，请花点时间自己验证它们。 例如，为了检查矩阵乘法的相关性，假设$A \in \mathbb{R}^ {m \times n}，$ $B \in \mathbb{R}^ {n \times p} $，$C \in \mathbb{R}^ {p \times q}$。 注意$AB \in \mathbb{R}^ {m \times p}$，所以$(AB)C \in \mathbb{R}^ {m \times q}$。 类似地，$BC \in \mathbb{R}^ {n \times q}$，所以$A(BC) \in \mathbb{R}^ {m \times q}$。 因此，所得矩阵的维度一致。 为了表明矩阵乘法是相关的，足以检查$(AB)C $的第$(i,j)$个元素是否等于$A(BC)$的第$(i,j)$个元素。 我们可以使用矩阵乘法的定义直接验证这一点：

$$
\begin{aligned}((A B) C)_{i j} &=\sum_{k=1}^{p}(A B)_{i k} C_{k j}=\sum_{k=1}^{p}\left(\sum_{l=1}^{n} A_{i l} B_{l k}\right) C_{k j} \\ &=\sum_{k=1}^{p}\left(\sum_{l=1}^{n} A_{i l} B_{l k} C_{k j}\right)=\sum_{l=1}^{n}\left(\sum_{k=1}^{p} A_{i l} B_{l k} C_{k j}\right) \\ &=\sum_{l=1}^{n} A_{i l}\left(\sum_{k=1}^{p} B_{l k} C_{k j}\right)=\sum_{l=1}^{n} A_{i l}(B C)_{l j}=(A(B C))_{i j} \end{aligned}
$$

### 3 运算和属性

在本节中，我们介绍矩阵和向量的几种运算和属性。 希望能够为您复习大量此类内容，这些笔记可以作为这些主题的参考。

#### 3.1 单位矩阵和对角矩阵

**单位矩阵**,$I \in \mathbb{R}^{n \times n} $，它是一个方阵，对角线的元素是1，其余元素都是0：
$$
I_{i j}=\left\{\begin{array}{ll}{1} & {i=j} \\ {0} & {i \neq j}\end{array}\right.
$$
对于所有$A \in \mathbb{R}^ {m \times n}$，有：
$$
AI = A = IA
$$
注意，在某种意义上，单位矩阵的表示法是不明确的，因为它没有指定$I$的维数。通常，$I$的维数是从上下文推断出来的，以便使矩阵乘法成为可能。 例如，在上面的等式中，$AI = A$中的I是$n\times n$矩阵，而$A = IA$中的$I$是$m\times m$矩阵。

对角矩阵是一种这样的矩阵：对角线之外的元素全为0。对角阵通常表示为：$D= diag(d_1, d_2, . . . , d_n)$，其中：
$$
D_{i j}=\left\{\begin{array}{ll}{d_{i}} & {i=j} \\ {0} & {i \neq j}\end{array}\right.
$$
很明显：单位矩阵$ I = diag(1, 1, . . . , 1)$。

#### 3.2 转置

矩阵的转置是指翻转矩阵的行和列。

给定一个矩阵：

$A \in \mathbb{R}^ {m \times n}$, 它的转置为$n \times m$的矩阵$A^T \in \mathbb{R}^ {n \times m}$ ，其中的元素为：
$$
(A^T)_{ij} = A_{ji}
$$
事实上，我们在描述行向量时已经使用了转置，因为列向量的转置自然是行向量。

转置的以下属性很容易验证：

- $(A^T )^T = A$
- $ (AB)^T = B^T A^T$
- $(A + B)^T = A^T + B^T$

#### 3.3 对称矩阵

如果$A =  A^T$，则矩阵$A \in \mathbb{R}^ {n \times n}$是对称矩阵。 如果$ A =  -  A^T$，它是反对称的。 很容易证明，对于任何矩阵$A \in \mathbb{R}^ {n \times n}$，矩阵$A  +  A^ T$是对称的，矩阵$A -A^T$是反对称的。 由此得出，任何方矩阵$A \in \mathbb{R}^ {n \times n}$可以表示为对称矩阵和反对称矩阵的和，所以：
$$
A=\frac{1}{2}(A+A^T)+\frac{1}{2}(A-A^T)
$$
上面公式的右边的第一个矩阵是对称矩阵，而第二个矩阵是反对称矩阵。 事实证明，对称矩阵在实践中用到很多，它们有很多很好的属性，我们很快就会看到它们。
通常将大小为$n$的所有对称矩阵的集合表示为$\mathbb{S}^n$，因此$A \in \mathbb{S}^n$意味着$A$是对称的$n\times n$矩阵;

#### 3.4 矩阵的迹

方矩阵$A \in \mathbb{R}^ {n \times n}$的迹，表示为$\operatorname{tr} (A)$（或者只是$\operatorname{tr} A$，如果括号显然是隐含的），是矩阵中对角元素的总和：
$$
\operatorname{tr} A=\sum_{i=1}^{n} A_{i i}
$$
如**CS229**讲义中所述，迹具有以下属性（如下所示）：

- 对于矩阵$A \in \mathbb{R}^ {n \times n}$，则：$\operatorname{tr}A =\operatorname{tr}A^T$

- 对于矩阵$A,B \in \mathbb{R}^ {n \times n}$，则：$\operatorname{tr}(A + B) = \operatorname{tr}A + \operatorname{tr}B$

- 对于矩阵$A \in \mathbb{R}^ {n \times n}$，$ t \in \mathbb{R}$，则：$\operatorname{tr}(tA) = t\operatorname{tr}A$.

- 对于矩阵 $A$, $B$，$AB$ 为方阵, 则：$\operatorname{tr}AB = \operatorname{tr}BA$

- 对于矩阵 $A$, $B$, $C$, $ABC$为方阵, 则：$\operatorname{tr}ABC = \operatorname{tr}BCA=\operatorname{tr}CAB$, 同理，更多矩阵的积也是有这个性质。

作为如何证明这些属性的示例，我们将考虑上面给出的第四个属性。 假设$A \in \mathbb{R}^ {m \times n}$和$B \in \mathbb{R}^ {n \times m}$（因此$AB \in \mathbb{R}^ {m \times m}$是方阵）。 观察到$BA \in \mathbb{R}^ {n \times n}$也是一个方阵，因此对它们进行迹的运算是有意义的。 要证明$\operatorname{tr}AB = \operatorname{tr}BA$，请注意：

$$
\begin{aligned} \operatorname{tr} A B &=\sum_{i=1}^{m}(A B)_{i i}=\sum_{i=1}^{m}\left(\sum_{j=1}^{n} A_{i j} B_{j i}\right) \\ &=\sum_{i=1}^{m} \sum_{j=1}^{n} A_{i j} B_{j i}=\sum_{j=1}^{n} \sum_{i=1}^{m} B_{j i} A_{i j} \\ &=\sum_{j=1}^{n}\left(\sum_{i=1}^{m} B_{j i} A_{i j}\right)=\sum_{j=1}^{n}(B A)_{j j}=\operatorname{tr} B A \end{aligned}
$$

这里，第一个和最后两个等式使用迹运算符和矩阵乘法的定义，重点在第四个等式，使用标量乘法的可交换性来反转每个乘积中的项的顺序，以及标量加法的可交换性和相关性，以便重新排列求和的顺序。

#### 3.5 范数

向量的范数$\|x\|$是非正式度量的向量的“长度” 。 例如，我们有常用的欧几里德或$\ell_{2}$范数，
$$
\|x\|_{2}=\sqrt{\sum_{i=1}^{n} x_{i}^{2}}
$$
注意：$\|x\|_{2}^{2}=x^{T} x$

更正式地，范数是满足4个属性的函数（$f : \mathbb{R}^{n} \rightarrow \mathbb{R}$）：

1. 对于所有的 $x \in \mathbb{R}^ {n}$, $f(x) \geq 0 $(非负).
2. 当且仅当$x = 0$ 时，$f(x) = 0$ (明确性).
3. 对于所有$x \in \mathbb{R}^ {n}$,$t\in \mathbb{R}$，则 $f(tx) = \left| t \right|f(x)$ (正齐次性).
4. 对于所有 $x,y \in \mathbb{R}^ {n}$, $f(x + y) \leq f(x) + f(y)$ (三角不等式)

其他范数的例子是$\ell_1$范数:
$$
\|x\|_{1}=\sum_{i=1}^{n}\left|x_{i}\right|
$$
和$\ell_{\infty }$范数：
$$
\|x\|_{\infty}=\max _{i}\left|x_{i}\right|
$$
事实上，到目前为止所提出的所有三个范数都是$\ell_p$范数族的例子，它们由实数$p \geq 1$参数化，并定义为：
$$
\|x\|_{p}=\left(\sum_{i=1}^{n}\left|x_{i}\right|^{p}\right)^{1 / p}
$$

也可以为矩阵定义范数，例如**Frobenius**范数:
$$
\|A\|_{F}=\sqrt{\sum_{i=1}^{m} \sum_{j=1}^{n} A_{i j}^{2}}=\sqrt{\operatorname{tr}\left(A^{T} A\right)}
$$
许多其他更多的范数，但它们超出了这个复习材料的范围。

#### 3.6 线性相关性和秩

一组向量${x_1,x_2, \cdots x_n} \in \mathbb{R}$， 如果没有向量可以表示为其余向量的线性组合，则称称该向量是线性无相关的。 相反，如果属于该组的一个向量可以表示为其余向量的线性组合，则称该向量是线性相关的。 也就是说，如果：
$$
x_{n}=\sum_{i=1}^{n-1} \alpha_{i} x_{i}
$$
对于某些标量值$\alpha_1,\cdots \alpha_n-1 \in \mathbb{R}$，要么向量$x_1,x_2, \cdots x_n$是线性相关的; 否则，向量是线性无关的。 例如，向量：
$$
x_{1}=\left[\begin{array}{l}{1} \\ {2} \\ {3}\end{array}\right] \quad x_{2}=\left[\begin{array}{c}{4} \\ {1} \\ {5}\end{array}\right] \quad x_{3}=\left[\begin{array}{c}{2} \\ {-3} \\ {-1}\end{array}\right]
$$
是线性相关的，因为：$x_3=-2x_1+x_2$。

矩阵$A  \in \mathbb{R}^{m \times n}$的**列秩**是构成线性无关集合的$A$的最大列子集的大小。 由于术语的多样性，这通常简称为$A$的线性无关列的数量。同样，行秩是构成线性无关集合的$A$的最大行数。 对于任何矩阵$A  \in \mathbb{R}^{m \times n}$，事实证明$A$的列秩等于$A$的行秩（尽管我们不会证明这一点），因此两个量统称为$A$的**秩**，用 $\text{rank}(A)$表示。 以下是秩的一些基本属性：

-   对于  $A  \in \mathbb{R}^{m \times n}$，$\text{rank}(A) \leq min(m, n)$，如果$ \text(A) = \text{min} (m, n)$，则： $A$ 被称作**满秩**。
-   对于  $A  \in \mathbb{R}^{m \times n}$， $\text{rank}(A) = \text{rank}(A^T)$
-   对于  $A  \in \mathbb{R}^{m \times n}$,$B  \in \mathbb{R}^{n \times p}$ ,$\text{rank}(AB) \leq \text{min} ( \text{rank}(A), \text{rank}(B))$
-   对于  $A,B \in \mathbb{R}^{m \times n}$，$\text{rank}(A + B) \leq \text{rank}(A) + \text{rank}(B)$

#### 3.7 方阵的逆

方阵$A  \in \mathbb{R}^{n \times n}$的倒数表示为$A^{-1}$，并且是这样的独特矩阵:
$$
A^{-1}A=I=AA^{-1}
$$
请注意，并非所有矩阵都具有逆。 例如，非方形矩阵根据定义没有逆。 然而，对于一些方形矩阵$A$，可能仍然存在$A^{-1}$可能不存在的情况。 特别是，如果$A^{-1}$存在，我们说$A$是**可逆**的或**非奇异**的，否则就是**不可逆**或**奇异**的。
为了使方阵A具有逆$A^{-1}$，则$A$必须是满秩。 我们很快就会发现，除了满秩之外，还有许多其它的充分必要条件。
以下是逆的属性; 假设$A,B  \in \mathbb{R}^{n \times n}$，而且是非奇异的：

-   $(A^{-1})^{-1} = A$
-   $(AB)^{-1} = B^{-1}A^{-1}$
-   $(A^{-1})^{T} =(A^{T})^{-1} $因此，该矩阵通常表示为$A^{-T}$。
作为如何使用逆的示例，考虑线性方程组，$Ax = b$，其中$A  \in \mathbb{R}^{n \times n}$，$x,b\in \mathbb{R}$， 如果$A$是非奇异的（即可逆的），那么$x = A^{-1}b$。 （如果$A  \in \mathbb{R}^{m \times n}$不是方阵，这公式还有用吗？）

#### 3.8 正交阵

如果 $x^Ty=0$，则两个向量$x,y\in \mathbb{R}^{n}$ 是**正交**的。如果$\|x\|_2=1$，则向量$x\in \mathbb{R}^{n}$ 被归一化。如果一个方阵$U\in \mathbb{R}^{n \times n}$的所有列彼此正交并被归一化（这些列然后被称为正交），则方阵$U$是正交阵（注意在讨论向量时的意义不一样）。

它可以从正交性和正态性的定义中得出:
$$
U^ TU = I = U U^T
$$

换句话说，正交矩阵的逆是其转置。 注意，如果$U$不是方阵 :即，$U\in \mathbb{R}^{m \times n}$，$n <m$  ，但其列仍然是正交的，则$U^TU = I$，但是$UU^T \neq I$。我们通常只使用术语"正交"来描述先前的情况 ，其中$U$是方阵。
正交矩阵的另一个好的特性是在具有正交矩阵的向量上操作不会改变其欧几里德范数，即:
$$
\|U x\|_{2}=\|x\|_{2}
$$
对于任何 $x\in \mathbb{R}$ , $U\in \mathbb{R}^{n}$是正交的。

#### 3.9 矩阵的值域和零空间

一组向量$\{x_{1}, \ldots x_{n}\}$是可以表示为$\{x_{1}, \ldots x_{n}\}$的线性组合的所有向量的集合。 即：
$$
\operatorname{span}\left(\left\{x_{1}, \ldots x_{n}\right\}\right)=\left\{v : v=\sum_{i=1}^{n} \alpha_{i} x_{i}, \quad \alpha_{i} \in \mathbb{R}\right\}
$$
可以证明，如果$\{x_{1}, \ldots x_{n}\}$是一组$n$个线性无关的向量，其中每个$x_i \in \mathbb{R}^{n}$，则$\text{span}(\{x_{1}, \ldots x_{n}\})=\mathbb{R}^{n}$。 换句话说，任何向量$v\in \mathbb{R}^{n}$都可以写成$x_1$到$x_n$的线性组合。

向量$y\in \mathbb{R}^{m}$投影到$\{x_{1}, \ldots x_{n}\}$（这里我们假设$x_i \in \mathbb{R}^{m}$）得到向量$v \in \operatorname{span}(\{x_{1}, \ldots, x_{n}\})$，由欧几里德范数$\|v  -  y\|_2$可以得知，这样$v$尽可能接近$y$。

我们将投影表示为$\operatorname{Proj}\left(y ;\left\{x_{1}, \ldots x_{n}\right\}\right)$，并且可以将其正式定义为:
$$
\operatorname{Proj}\left(y ;\left\{x_{1}, \ldots x_{n}\right\}\right)=\operatorname{argmin}_{v \in \operatorname{span}\left(\left\{x_{1}, \ldots, x_{n}\right\}\right)}\|y-v\|_{2}
$$
矩阵$A\in \mathbb{R}^{m \times n}$的值域（有时也称为列空间），表示为$\mathcal{R}(A)$，是$A$列的跨度。换句话说，
$$
\mathcal{R}(A)=\left\{v \in \mathbb{R}^{m} : v=A x, x \in \mathbb{R}^{n}\right\}
$$
做一些技术性的假设（即$A$是满秩且$n <m$），向量$y \in \mathbb{R}^{m}$到$A$的范围的投影由下式给出:
$$
\operatorname{Proj}(y ; A)=\operatorname{argmin}_{v \in \mathcal{R}(A)}\|v-y\|_{2}=A\left(A^{T} A\right)^{-1} A^{T} y
$$
这个最后的方程应该看起来非常熟悉，因为它几乎与我们在课程中（我们将很快再次得出）得到的公式：用于参数的最小二乘估计一样。 看一下投影的定义，显而易见，这实际上是我们在最小二乘问题中最小化的目标（除了范数的平方这里有点不一样，这不会影响找到最优解），所以这些问题自然是非常相关的。 

当$A$只包含一列时，$a \in \mathbb{R}^{m}$，这给出了向量投影到一条线上的特殊情况：
$$
\operatorname{Proj}(y ; a)=\frac{a a^{T}}{a^{T} a} y
$$
一个矩阵$A\in \mathbb{R}^{m \times n}$的零空间 $\mathcal{N}(A)$ 是所有乘以$A$时等于0向量的集合，即：
$$
\mathcal{N}(A)=\left\{x \in \mathbb{R}^{n} : A x=0\right\}
$$
注意，$\mathcal{R}(A)$中的向量的大小为$m$，而 $\mathcal{N}(A)$ 中的向量的大小为$n$，因此$\mathcal{R}(A^T)$和 $\mathcal{N}(A)$ 中的向量的大小均为$\mathbb{R}^{n}$。 事实上，还有很多例子。 证明：
$$
\left\{w : w=u+v, u \in \mathcal{R}\left(A^{T}\right), v \in \mathcal{N}(A)\right\}=\mathbb{R}^{n} \text { and } \mathcal{R}\left(A^{T}\right) \cap \mathcal{N}(A)=\{\mathbf{0}\}
$$
换句话说，$\mathcal{R}(A^T)$和 $\mathcal{N}(A)$ 是不相交的子集，它们一起跨越$\mathbb{R}^{n}$的整个空间。 这种类型的集合称为**正交补**，我们用$\mathcal{R}(A^T)= \mathcal{N}(A)^{\perp}$表示。

#### 3.10 行列式

一个方阵$A  \in \mathbb{R}^{n \times n}$的行列式是函数$\text {det}$：$\mathbb{R}^{n \times n} \rightarrow \mathbb{R}^{n} $，并且表示为$\left| A \right|$。 或者$\text{det} A$（有点像迹运算符，我们通常省略括号）。 从代数的角度来说，我们可以写出一个关于$A$行列式的显式公式。 因此，我们首先提供行列式的几何解释，然后探讨它的一些特定的代数性质。

给定一个矩阵：
$$
\left[\begin{array}{cccc}{-} & {a_{1}^{T}}  & {-} \\ {-} & {a_{2}^{T}} & {-} \\ {} & {\vdots} & {} \\  {-} & {a_{n}^{T}} & {-}\end{array}\right]
$$

考虑通过采用$A$行向量$a_{1}, \ldots a_{n}\in  \mathbb{R}^{n}$的所有可能线性组合形成的点$S \subset \mathbb{R}^{n}$的集合，其中线性组合的系数都在0和1之间; 也就是说，集合$S$是$\text{span}(\{a_{1}, \ldots a_{n}\})$受到系数$a_{1}, \ldots a_{n}$的限制的线性组合，$\alpha_1, \cdots ,\alpha_n$满足$0 \leq \alpha_{i} \leq 1, i=1, \ldots, n$。从形式上看，
$$
S=\left\{v \in \mathbb{R}^{n} : v=\sum_{i=1}^{n} \alpha_{i} a_{i} \text { where } 0 \leq \alpha_{i} \leq 1, i=1, \ldots, n\right\}
$$
事实证明，$A$的行列式的绝对值是对集合$S$的“体积”的度量。

比方说：一个$2 \times2$的矩阵(4)：
$$
A=\left[\begin{array}{ll}{1} & {3} \\ {3} & {2}\end{array}\right]
$$
它的矩阵的行是：
$$
a_{1}=\left[\begin{array}{l}{1} \\ {3}\end{array}\right] \quad a_{2}=\left[\begin{array}{l}{3} \\ {2}\end{array}\right]
$$
对应于这些行对应的集合$S$如图1所示。对于二维矩阵，$S$通常具有平行四边形的形状。 在我们的例子中，行列式的值是$\left| A \right| = -7$（可以使用本节后面显示的公式计算），因此平行四边形的面积为7。（请自己验证！）

在三维中，集合$S$对应于一个称为平行六面体的对象（一个有倾斜边的三维框，这样每个面都有一个平行四边形）。行定义$S$的$3×3$矩阵S的行列式的绝对值给出了平行六面体的三维体积。在更高的维度中，集合$S$是一个称为$n$维平行切的对象。

![](images/fig1.png)

图1：（4）中给出的$2×2$矩阵$A$的行列式的图示。 这里，$a_1$和$a_2$是对应于$A$行的向量，并且集合$S$对应于阴影区域（即，平行四边形）。 这个行列式的绝对值，$\left| \text{det} A \right| = 7$，即平行四边形的面积。

在代数上，行列式满足以下三个属性（所有其他属性都遵循这些属性，包括通用公式）：

1. 恒等式的行列式为1, $\left| I \right|= 1$（几何上，单位超立方体的体积为1）。

2. 给定一个矩阵 $A  \in \mathbb{R}^{n \times n}$, 如果我们将$A$中的一行乘上一个标量$t  \in \mathbb{R}$，那么新矩阵的行列式是$t\left| A \right|$
$$
\left|\left[\begin{array}{ccc}{-} & {t a_{1}^{T}} & {-} \\ {-} & {a_{2}^{T}} & {-} \\ {} & {\vdots} & {} \\ {} & {a_{m}^{T}} & {-}\end{array}\right]\right|=t|A|
$$
几何上，将集合$S$的一个边乘以系数$t$，体积也会增加一个系数$t$。

3. 如果我们交换任意两行在$a_i^T$和$a_j^T$，那么新矩阵的行列式是$-\left| A \right|$，例如：
$$
\left|\left[\begin{array}{ccc}{-} & {a_{2}^{T}} & {-} \\ {-} & {a_{1}^{T}} & {-} \\ {} & {\vdots} & {} \\ {-} & {a_{m}^{T}} & {-}\end{array}\right]\right|=-|A|
$$
你一定很奇怪，满足上述三个属性的函数的存在并不多。事实上，这样的函数确实存在，而且是唯一的（我们在这里不再证明了）。

从上述三个属性中得出的几个属性包括：

-   对于 $A  \in \mathbb{R}^{n \times n}$, $\left| A \right| = \left| A^T \right|$
-   对于 $A,B \in \mathbb{R}^{n \times n}$, $\left| AB \right|= \left| A \right|\left| B \right|$
-   对于 $A  \in \mathbb{R}^{n \times n}$, 有且只有当$A$是奇异的（比如不可逆） ，则：$\left| A \right|= 0$
-   对于 $A  \in \mathbb{R}^{n \times n}$ 同时，$A$为非奇异的，则：$\left| A ^{−1}\right| = 1/\left| A \right|$

在给出行列式的一般定义之前，我们定义，对于$A  \in \mathbb{R}^{n \times n}$，$A_{\backslash i, \backslash j}\in \mathbb{R}^{(n-1) \times (n-1)}$是由于删除第$i$行和第$j$列而产生的矩阵。 行列式的一般（递归）公式是：
$$
\begin{aligned}|A| &=\sum_{i=1}^{n}(-1)^{i+j} a_{i j}\left|A_{\backslash i, \backslash j}\right| \quad(\text { for any } j \in 1, \ldots, n) \\ &=\sum_{j=1}^{n}(-1)^{i+j} a_{i j}\left|A_{\backslash i, \backslash j}\right| \quad(\text { for any } i \in 1, \ldots, n) \end{aligned}
$$
对于 $A  \in \mathbb{R}^{1 \times 1}$，初始情况为$\left| A \right|= a_{11}$。如果我们把这个公式完全展开为 $A  \in \mathbb{R}^{n \times n}$，就等于$n!$（$n$阶乘）不同的项。因此，对于大于$3×3$的矩阵，我们几乎没有明确地写出完整的行列式方程。然而，$3×3$大小的矩阵的行列式方程是相当常见的，建议好好地了解它们：
$$
\left|\left[a_{11}\right]\right|=a_{11}
$$

$$
\left|\left[\begin{array}{ll}{a_{11}} & {a_{12}} \\ {a_{21}} & {a_{22}}\end{array}\right]\right|=a_{11} a_{22}-a_{12} a_{21}
$$

$$
\left|\left[\begin{array}{l}{a_{11}} & {a_{12}} & {a_{13}} \\ {a_{21}} & {a_{22}} & {a_{23}} \\ {a_{31}} & {a_{32}} & {a_{33}}\end{array}\right]\right|=\quad \begin{array}{c}{a_{11} a_{22} a_{33}+a_{12} a_{23} a_{31}+a_{13} a_{21} a_{32}} \\\quad \quad {-a_{11} a_{23} a_{32}-a_{12} a_{21} a_{33}-a_{13} a_{22} a_{31}} \\ {}\end{array}
$$
矩阵$A  \in \mathbb{R}^{n \times n}$的经典伴随矩阵（通常称为伴随矩阵）表示为$\operatorname{adj}(A)$，并定义为：
$$
\operatorname{adj}(A) \in \mathbb{R}^{n \times n}, \quad(\operatorname{adj}(A))_{i j}=(-1)^{i+j}\left|A_{\backslash j, \backslash i}\right|
$$
（注意索引$A_{\backslash j, \backslash i}$中的变化）。可以看出，对于任何非奇异$A  \in \mathbb{R}^{n \times n}$，
$$
A^{-1}=\frac{1}{|A|} \operatorname{adj}(A)
$$

虽然这是一个很好的“显式”的逆矩阵公式，但我们应该注意，从数字上讲，有很多更有效的方法来计算逆矩阵。

#### 3.11 二次型和半正定矩阵

给定方矩阵$A  \in \mathbb{R}^{n \times n}$和向量$x \in \mathbb{R}^{n}$，标量值$x^T Ax$被称为二次型。 写得清楚些，我们可以看到：
$$
x^{T} A x=\sum_{i=1}^{n} x_{i}(A x)_{i}=\sum_{i=1}^{n} x_{i}\left(\sum_{j=1}^{n} A_{i j} x_{j}\right)=\sum_{i=1}^{n} \sum_{j=1}^{n} A_{i j} x_{i} x_{j}
$$
注意：
$$
x^{T} A x=\left(x^{T} A x\right)^{T}=x^{T} A^{T} x=x^{T}\left(\frac{1}{2} A+\frac{1}{2} A^{T}\right) x
$$
第一个等号的是因为是标量的转置与自身相等，而第二个等号是因为是我们平均两个本身相等的量。 由此，我们可以得出结论，只有$A$的对称部分有助于形成二次型。 出于这个原因，我们经常隐含地假设以二次型出现的矩阵是对称阵。
我们给出以下定义：

-   对于所有非零向量$x \in \mathbb{R}^n$，$x^TAx>0$，对称阵$A \in \mathbb{S}^n$为**正定**（**positive definite,PD**）。这通常表示为$A\succ0$（或$A>0$），并且通常将所有正定矩阵的集合表示为$\mathbb{S}_{++}^n$。

-   对于所有向量$x^TAx\geq 0$，对称矩阵$A \in \mathbb{S}^n$是**半正定**(**positive semidefinite ,PSD**)。 这写为（或$A \succeq 0$仅$A≥0$），并且所有半正定矩阵的集合通常表示为$\mathbb{S}_+^n$。

-   同样，对称矩阵$A \in \mathbb{S}^n$是**负定**（**negative definite,ND**），如果对于所有非零$x \in \mathbb{R}^n$，则$x^TAx <0$表示为$A\prec0$（或$A <0$）。
  
-   类似地，对称矩阵$A \in \mathbb{S}^n$是**半负定**(**negative semidefinite,NSD**），如果对于所有$x \in \mathbb{R}^n$，则$x^TAx \leq 0$表示为$A\preceq 0$（或$A≤0$）。

-   最后，对称矩阵$A \in \mathbb{S}^n$是**不定**的，如果它既不是正半定也不是负半定，即，如果存在$x_1,x_2 \in \mathbb{R}^n$，那么$x_1^TAx_1>0$且$x_2^TAx_2<0$。

很明显，如果$A$是正定的，那么$−A$是负定的，反之亦然。同样，如果$A$是半正定的，那么$−A$是是半负定的，反之亦然。如果果$A$是不定的，那么$−A$是也是不定的。

正定矩阵和负定矩阵的一个重要性质是它们总是满秩，因此是可逆的。为了了解这是为什么，假设某个矩阵$A \in \mathbb{S}^n$不是满秩。然后，假设$A$的第$j$列可以表示为其他$n-1$列的线性组合：
$$
a_{j}=\sum_{i \neq j} x_{i} a_{i}
$$
对于某些$x_1,\cdots x_{j-1},x_{j + 1} ,\cdots ,x_n\in \mathbb{R}$。设$x_j = -1$，则：
$$
Ax=\sum_{i \neq j} x_{i} a_{i}=0
$$
但这意味着对于某些非零向量$x$，$x^T Ax = 0$，因此$A$必须既不是正定也不是负定。如果$A$是正定或负定，则必须是满秩。
最后，有一种类型的正定矩阵经常出现，因此值得特别提及。 给定矩阵$A  \in \mathbb{R}^{m \times n}$（不一定是对称或偶数平方），矩阵$G = A^T A$（有时称为**Gram矩阵**）总是半正定的。 此外，如果$m\geq n$（同时为了方便起见，我们假设$A$是满秩），则$G = A^T A$是正定的。

#### 3.12 特征值和特征向量

给定一个方阵$A \in\mathbb{R}^{n\times n}$，我们认为在以下条件下，$\lambda \in\mathbb{C}$是$A$的**特征值**，$x\in\mathbb{C}^n$是相应的**特征向量**：

$$
Ax=\lambda x,x \ne 0
$$

直观地说，这个定义意味着将$A$乘以向量$x$会得到一个新的向量，该向量指向与$x$相同的方向，但按系数$\lambda$缩放。值得注意的是，对于任何特征向量$x\in\mathbb{C}^n$和标量$t\in\mathbb{C}$，$A(cx)=cAx=c\lambda x=\lambda(cx)$，$cx$也是一个特征向量。因此，当我们讨论与$\lambda$相关的**特征向量**时，我们通常假设特征向量被标准化为长度为1（这仍然会造成一些歧义，因为$x$和$−x$都是特征向量，但我们必须接受这一点）。

我们可以重写上面的等式来说明$(\lambda,x)$是$A$的特征值和特征向量的组合：
$$
(\lambda I-A)x=0,x \ne 0
$$
但是$(\lambda I-A)x=0$只有当$(\lambda I-A)$有一个非空零空间时，同时$(\lambda I-A)$是奇异的，$x$才具有非零解，即：
$$
|(\lambda I-A)|=0
$$
现在，我们可以使用行列式的先前定义将表达式$|(\lambda I-A)|$扩展为$\lambda$中的（非常大的）多项式，其中，$\lambda$的度为$n$。它通常被称为矩阵$A$的特征多项式。

然后我们找到这个特征多项式的$n$（可能是复数）根，并用$\lambda_1,\cdots,\lambda_n$表示。这些都是矩阵$A$的特征值，但我们注意到它们可能不明显。为了找到特征值$\lambda_i$对应的特征向量，我们只需解线性方程$(\lambda I-A)x=0$，因为$(\lambda I-A)$是奇异的，所以保证有一个非零解（但也可能有多个或无穷多个解）。

应该注意的是，这不是实际用于数值计算特征值和特征向量的方法（记住行列式的完全展开式有$n!$项），这是一个数学上的争议。

以下是特征值和特征向量的属性（所有假设在$A \in\mathbb{R}^{n\times n}$具有特征值$\lambda_1,\cdots,\lambda_n$的前提下）：

- $A$的迹等于其特征值之和
  $$
  \operatorname{tr} A=\sum_{i=1}^{n} \lambda_{i}
  $$

- $A$的行列式等于其特征值的乘积
  $$
  |A|=\prod_{i=1}^{n} \lambda_{i}
  $$

- $A$的秩等于$A$的非零特征值的个数
  
- 假设$A$非奇异，其特征值为$\lambda$和特征向量为$x$。那么$1/\lambda$是具有相关特征向量$x$的$A^{-1}$的特征值，即$A^{-1}x=(1/\lambda)x$。（要证明这一点，取特征向量方程，$Ax=\lambda x$，两边都左乘$A^{-1}$）
  
- 对角阵的特征值$d=diag(d_1，\cdots,d_n)$实际上就是对角元素$d_1，\cdots,d_n$

#### 3.13 对称矩阵的特征值和特征向量

通常情况下，一般的方阵的特征值和特征向量的结构可以很细微地表示出来。
值得庆幸的是，在机器学习的大多数场景下，处理对称实矩阵就足够了，其处理的对称实矩阵的特征值和特征向量具有显着的特性。

在本节中，我们假设$A$是实对称矩阵, 具有以下属性：

1. $A$的所有特征值都是实数。 我们用用$\lambda_1,\cdots,\lambda_n$表示。

2. 存在一组特征向量$u_1，\cdots u_n$，对于所有$i$，$u_i$是具有特征值$\lambda_{i}$和$b$的特征向量。$u_1，\cdots u_n$是单位向量并且彼此正交。

设$U$是包含$u_i$作为列的正交矩阵：
$$
U=\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {u_{1}} & {u_{2}} & {\cdots} & {u_{n}} \\ { |} & { |} & {} & { |}\end{array}\right]
$$
设$\Lambda= diag(\lambda_1,\cdots,\lambda_n)$是包含$\lambda_1,\cdots,\lambda_n$作为对角线上的元素的对角矩阵。 使用2.3节的方程（2）中的矩阵 - 矩阵向量乘法的方法，我们可以验证：
$$
A U=\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {A u_{1}} & {A u_{2}} & {\cdots} & {A u_{n}} \\ { |} & { |} & {} & { |}\end{array}\right]=\left[\begin{array}{ccc}{ |} & { |} & { |} & { |}\\ {\lambda_{1} u_{1}} & {\lambda_{2} u_{2}} & {\cdots} & {\lambda_{n} u_{n}} \\ { |} & { |} & {|} & { |}\end{array}\right]=U \operatorname{diag}\left(\lambda_{1}, \ldots, \lambda_{n}\right)=U \Lambda
$$
考虑到正交矩阵$U$满足$UU^T=I$，利用上面的方程，我们得到：
$$
A=AUU^T=U\Lambda U^T
$$

这种$A$的新的表示形式为$U\Lambda U^T$，通常称为矩阵$A$的对角化。术语对角化是这样来的：通过这种表示，我们通常可以有效地将对称矩阵$A$视为对角矩阵 , 这更容易理解。关于由特征向量$U$定义的基础， 我们将通过几个例子详细说明。

**背景知识**：代表另一个基的向量。

任何正交矩阵$U=\left[\begin{array}{cccc}{ |} & { |} & {} & { |} \\ {u_{1}} & {u_{2}} & {\cdots} & {u_{n}} \\ { |} & { |} & {} & { |}\end{array}\right]$定义了一个新的属于$\mathbb {R}^{n}$的基（坐标系），意义如下：对于任何向量$x \in\mathbb{R}^{n}$都可以表示为$u_1，\cdots u_n$的线性组合，其系数为$x_1,\cdots x_n$：

$$
x=\hat x_1u_1+\cdots +\cdots \hat x_nu_n=U\hat x
$$

在第二个等式中，我们使用矩阵和向量相乘的方法。 实际上，这种$\hat x$是唯一存在的:
$$
x=U \hat{x} \Leftrightarrow U^{T} x=\hat{x}
$$
换句话说，向量$\hat x=U^Tx$可以作为向量$x$的另一种表示，与$U$定义的基有关。

**“对角化”矩阵向量乘法**。 通过上面的设置，我们将看到左乘矩阵$A$可以被视为左乘以对角矩阵关于特征向量的基。 假设$x$是一个向量，$\hat x$表示$U$的基。设$z=Ax$为矩阵向量积。现在让我们计算关于$U$的基$z$：
然后，再利用$UU^T=U^T=I$和方程$A=AUU^T=U\Lambda U^T$，我们得到：
$$
\hat{z}=U^{T} z=U^{T} A x=U^{T} U \Lambda U^{T} x=\Lambda \hat{x}=\left[\begin{array}{c}{\lambda_{1} \hat{x}_{1}} \\ {\lambda_{2} \hat{x}_{2}} \\ {\vdots} \\ {\lambda_{n} \hat{x}_{n}}\end{array}\right]
$$
我们可以看到，原始空间中的左乘矩阵$A$等于左乘对角矩阵$\Lambda$相对于新的基，即仅将每个坐标缩放相应的特征值。
在新的基上，矩阵多次相乘也变得简单多了。例如，假设$q=AAAx$。根据$A$的元素导出$q$的分析形式，使用原始的基可能是一场噩梦，但使用新的基就容易多了：
$$
\hat{q}=U^{T} q=U^{T} AAA x=U^{T} U \Lambda U^{T} U \Lambda U^{T} U \Lambda U^{T} x=\Lambda^{3} \hat{x}=\left[\begin{array}{c}{\lambda_{1}^{3} \hat{x}_{1}} \\ {\lambda_{2}^{3} \hat{x}_{2}} \\ {\vdots} \\ {\lambda_{n}^{3} \hat{x}_{n}}\end{array}\right]
$$
**“对角化”二次型**。作为直接的推论，二次型$x^TAx$也可以在新的基上简化。
$$
x^{T} A x=x^{T} U \Lambda U^{T} x=\hat{x} \Lambda \hat{x}=\sum_{i=1}^{n} \lambda_{i} \hat{x}_{i}^{2}
$$
(回想一下，在旧的表示法中，$x^{T} A x=\sum_{i=1, j=1}^{n} x_{i} x_{j} A_{i j}$涉及一个$n^2$项的和，而不是上面等式中的$n$项。)利用这个观点，我们还可以证明矩阵$A$的正定性完全取决于其特征值的符号：

1. 如果所有的$\lambda_i>0$，则矩阵$A$正定的，因为对于任意的$\hat x \ne 0$,$x^{T} A x=\sum_{i=1}^{n} \lambda_{i} \hat{x}_{i}^{2}>0$
2. 如果所有的$\lambda_i\geq 0$，则矩阵$A$是为正半定，因为对于任意的$\hat x $,$x^{T} A x=\sum_{i=1}^{n} \lambda_{i} \hat{x}_{i}^{2} \geq 0$
3. 同样，如果所有$\lambda_i<0$或$\lambda_i\leq 0$，则矩阵$A$分别为负定或半负定。
4. 最后，如果$A$同时具有正特征值和负特征值，比如λ$\lambda_i>0$和$\lambda_j<0$，那么它是不定的。这是因为如果我们让$\hat x$满足$\hat x_i=1$和$\hat x_k=0$，同时所有的$k\ne i$，那么$x^{T} A x=\sum_{i=1}^{n} \lambda_{i} \hat{x}_{i}^{2}>0$ ,我们让$\hat x$满足$\hat x_i=1$和$\hat x_k=0$，同时所有的$k\ne i$，那么$x^{T} A x=\sum_{i=1}^{n} \lambda_{i} \hat{x}_{i}^{2}<0$ 

特征值和特征向量经常出现的应用是最大化矩阵的某些函数。特别是对于矩阵$A \in \mathbb{S}^{n}$，考虑以下最大化问题：
$$
\max _{x \in \mathbb{R}^{n}} \ x^{T} A x=\sum_{i=1}^{n} \lambda_{i} \hat{x}_{i}^{2} \quad \text { subject to }\|x\|_{2}^{2}=1
$$
也就是说，我们要找到（范数1）的向量，它使二次型最大化。假设特征值的阶数为$\lambda_1 \geq \lambda _2 \geq \cdots \lambda_n$，此优化问题的最优值为$\lambda_1$，且与$\lambda_1$对应的任何特征向量$u_1$都是最大值之一。（如果$\lambda_1 > \lambda_2$，那么有一个与特征值$\lambda_1$对应的唯一特征向量，它是上面那个优化问题的唯一最大值。）
我们可以通过使用对角化技术来证明这一点：注意，通过公式$\|U x\|_{2}=\|x\|_{2}$推出$\|x\|_{2}=\|\hat{x}\|_{2}$，并利用公式：

$x^{T} A x=x^{T} U \Lambda U^{T} x=\hat{x} \Lambda \hat{x}=\sum_{i=1}^{n} \lambda_{i} \hat{x}_{i}^{2}$，我们可以将上面那个优化问题改写为：
$$
\max _{\hat{x} \in \mathbb{R}^{n}}\ \hat{x}^{T} \Lambda \hat{x}=\sum_{i=1}^{n} \lambda_{i} \hat{x}_{i}^{2} \quad \text { subject to }\|\hat{x}\|_{2}^{2}=1
$$
然后，我们得到目标的上界为$\lambda_1$：
$$
\hat{x}^{T} \Lambda \hat{x}=\sum_{i=1}^{n} \lambda_{i} \hat{x}_{i}^{2} \leq \sum_{i=1}^{n} \lambda_{1} \hat{x}_{i}^{2}=\lambda_{1}
$$
此外，设置$\hat{x}=\left[\begin{array}{c}{1} \\ {0} \\ {\vdots} \\ {0}\end{array}\right]$可让上述等式成立，这与设置$x=u_1$相对应。

### 4.矩阵微积分
虽然前面章节中的主题通常包含在线性代数的标准课程中，但似乎很少涉及（我们将广泛使用）的一个主题是微积分扩展到向量设置展。尽管我们使用的所有实际微积分都是相对微不足道的，但是符号通常会使事情看起来比实际困难得多。 在本节中，我们将介绍矩阵微积分的一些基本定义，并提供一些示例。

#### 4.1 梯度
假设$f: \mathbb{R}^{m \times n} \rightarrow \mathbb{R}$是将维度为$m \times n$的矩阵$A\in \mathbb{R}^{m \times n}$作为输入并返回实数值的函数。 然后$f$的梯度（相对于$A\in \mathbb{R}^{m \times n}$）是偏导数矩阵，定义如下：
$$
\nabla_{A} f(A) \in \mathbb{R}^{m \times n}=\left[\begin{array}{cccc}{\frac{\partial f(A)}{\partial A_{11}}} & {\frac{\partial f(A)}{\partial A_{12}}} & {\cdots} & {\frac{\partial f(A)}{\partial A_{1n}}} \\ {\frac{\partial f(A)}{\partial A_{21}}} & {\frac{\partial f(A)}{\partial A_{22}}} & {\cdots} & {\frac{\partial f(A)}{\partial A_{2 n}}} \\ {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ {\frac{\partial f(A)}{\partial A_{m 1}}} & {\frac{\partial f(A)}{\partial A_{m 2}}} & {\cdots} & {\frac{\partial f(A)}{\partial A_{m n}}}\end{array}\right]
$$
即，$m \times n$矩阵:
$$
\left(\nabla_{A} f(A)\right)_{i j}=\frac{\partial f(A)}{\partial A_{i j}}
$$
请注意，$\nabla_{A} f(A) $的维度始终与$A$的维度相同。特殊情况，如果$A$只是向量$A\in \mathbb{R}^{n}$，则
$$
\nabla_{x} f(x)=\left[\begin{array}{c}{\frac{\partial f(x)}{\partial x_{1}}} \\ {\frac{\partial f(x)}{\partial x_{2}}} \\ {\vdots} \\ {\frac{\partial f(x)}{\partial x_{n}}}\end{array}\right]
$$
重要的是要记住，只有当函数是实值时，即如果函数返回标量值，才定义函数的梯度。例如，$A\in \mathbb{R}^{m \times n}$相对于$x$，我们不能取$Ax$的梯度，因为这个量是向量值。
它直接从偏导数的等价性质得出：

- $\nabla_{x}(f(x)+g(x))=\nabla_{x} f(x)+\nabla_{x} g(x)$

- 对于$t \in \mathbb{R}$ ，$\nabla_{x}(t f(x))=t \nabla_{x} f(x)$

原则上，梯度是偏导数对多变量函数的自然延伸。然而，在实践中，由于符号的原因，使用梯度有时是很困难的。例如，假设$A\in \mathbb{R}^{m \times n}$是一个固定系数矩阵，假设$b\in \mathbb{R}^{m}$是一个固定系数向量。设$f: \mathbb{R}^{m \times n} \rightarrow \mathbb{R}$为$f(z)=z^Tz$定义的函数，因此$\nabla_{z}f(z)=2z$。但现在考虑表达式，
$$
\nabla f(Ax)
$$
该表达式应该如何解释？ 至少有两种可能性：
1.在第一个解释中，回想起$\nabla_{z}f(z)=2z$。 在这里，我们将$\nabla f(Ax)$解释为评估点$Ax$处的梯度，因此:

$$
\nabla f(A x)=2(A x)=2 A x \in \mathbb{R}^{m}
$$
2.在第二种解释中，我们将数量$f(Ax)$视为输入变量$x$的函数。 更正式地说，设$g(x) =f(Ax)$。 然后在这个解释中:
$$
\nabla f(A x)=\nabla_{x} g(x) \in \mathbb{R}^{n}
$$

在这里，我们可以看到这两种解释确实不同。 一种解释产生$m$维向量作为结果，而另一种解释产生$n$维向量作为结果！ 我们怎么解决这个问题？

这里，关键是要明确我们要区分的变量。
在第一种情况下，我们将函数$f$与其参数$z$进行区分，然后替换参数$Ax$。
在第二种情况下，我们将复合函数$g(x)=f(Ax)$直接与$x$进行微分。

我们将第一种情况表示为$\nabla zf(Ax)$，第二种情况表示为$\nabla xf(Ax)$。

保持符号清晰是非常重要的，以后完成课程作业时候你就会发现。

#### 4.2 黑塞矩阵

假设$f: \mathbb{R}^{n} \rightarrow \mathbb{R}$是一个函数，它接受$\mathbb{R}^{n}$中的向量并返回实数。那么关于$x$的**黑塞矩阵**（也有翻译作海森矩阵），写做：$\nabla_x ^2 f(A x)$，或者简单地说，$H$是$n \times n$矩阵的偏导数：
$$
\nabla_{x}^{2} f(x) \in \mathbb{R}^{n \times n}=\left[\begin{array}{cccc}{\frac{\partial^{2} f(x)}{\partial x_{1}^{2}}} & {\frac{\partial^{2} f(x)}{\partial x_{1} \partial x_{2}}} & {\cdots} & {\frac{\partial^{2} f(x)}{\partial x_{1} \partial x_{n}}} \\ {\frac{\partial^{2} f(x)}{\partial x_{2} \partial x_{1}}} & {\frac{\partial^{2} f(x)}{\partial x_{2}^{2}}} & {\cdots} & {\frac{\partial^{2} f(x)}{\partial x_{2} \partial x_{n}}} \\ {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ {\frac{\partial^{2} f(x)}{\partial x_{n} \partial x_{1}}} & {\frac{\partial^{2} f(x)}{\partial x_{n} \partial x_{2}}} & {\cdots} & {\frac{\partial^{2} f(x)}{\partial x_{n}^{2}}}\end{array}\right]
$$
换句话说，$\nabla_{x}^{2} f(x) \in \mathbb{R}^{n \times n}$，其：

$$
\left(\nabla_{x}^{2} f(x)\right)_{i j}=\frac{\partial^{2} f(x)}{\partial x_{i} \partial x_{j}}
$$
注意：黑塞矩阵通常是对称阵：

$$
\frac{\partial^{2} f(x)}{\partial x_{i} \partial x_{j}}=\frac{\partial^{2} f(x)}{\partial x_{j} \partial x_{i}}
$$
与梯度相似，只有当$f(x)$为实值时才定义黑塞矩阵。

很自然地认为梯度与向量函数的一阶导数的相似，而黑塞矩阵与二阶导数的相似（我们使用的符号也暗示了这种关系）。 这种直觉通常是正确的，但需要记住以下几个注意事项。
首先，对于一个变量$f: \mathbb{R} \rightarrow \mathbb{R}$的实值函数，它的基本定义：二阶导数是一阶导数的导数，即：
$$
\frac{\partial^{2} f(x)}{\partial x^{2}}=\frac{\partial}{\partial x} \frac{\partial}{\partial x} f(x)
$$
然而，对于向量的函数，函数的梯度是一个向量，我们不能取向量的梯度，即:
$$
\nabla_{x} \nabla_{x} f(x)=\nabla_{x}\left[\begin{array}{c}{\frac{\partial f(x)}{\partial x_{1}}} \\ {\frac{\partial f(x)}{\partial x_{2}}} \\ {\vdots} \\ {\frac{\partial f(x)}{\partial x_{n}}}\end{array}\right]
$$

上面这个表达式没有意义。 因此，黑塞矩阵不是梯度的梯度。 然而，下面这种情况却这几乎是正确的：如果我们看一下梯度$\left(\nabla_{x} f(x)\right)_{i}=\partial f(x) / \partial x_{i}$的第$i$个元素，并取关于于$x$的梯度我们得到：
$$
\nabla_{x} \frac{\partial f(x)}{\partial x_{i}}=\left[\begin{array}{c}{\frac{\partial^{2} f(x)}{\partial x_{i} \partial x_{1}}} \\ {\frac{\partial^{2} f(x)}{\partial x_{2} \partial x_{2}}} \\ {\vdots} \\ {\frac{\partial f(x)}{\partial x_{i} \partial x_{n}}}\end{array}\right]
$$

这是黑塞矩阵第$i$行（列）,所以：
$$
\nabla_{x}^{2} f(x)=\left[\nabla_{x}\left(\nabla_{x} f(x)\right)_{1} \quad \nabla_{x}\left(\nabla_{x} f(x)\right)_{2} \quad \cdots \quad \nabla_{x}\left(\nabla_{x} f(x)\right)_{n}\right]
$$
简单地说：我们可以说由于：$\nabla_{x}^{2} f(x)=\nabla_{x}\left(\nabla_{x} f(x)\right)^{T}$，只要我们理解，这实际上是取$\nabla_{x} f(x)$的每个元素的梯度，而不是整个向量的梯度。

最后，请注意，虽然我们可以对矩阵$A\in \mathbb{R}^{n}$取梯度，但对于这门课，我们只考虑对向量$x \in \mathbb{R}^{n}$取黑塞矩阵。
这会方便很多（事实上，我们所做的任何计算都不要求我们找到关于矩阵的黑森方程），因为关于矩阵的黑塞方程就必须对矩阵所有元素求偏导数$\partial^{2} f(A) /\left(\partial A_{i j} \partial A_{k \ell}\right)$，将其表示为矩阵相当麻烦。

#### 4.3 二次函数和线性函数的梯度和黑塞矩阵

现在让我们尝试确定几个简单函数的梯度和黑塞矩阵。 应该注意的是，这里给出的所有梯度都是**CS229**讲义中给出的梯度的特殊情况。

对于$x \in \mathbb{R}^{n}$, 设$f(x)=b^Tx$  的某些已知向量$b \in \mathbb{R}^{n}$ ，则：

$$
f(x)=\sum_{i=1}^{n} b_{i} x_{i}
$$
所以：
$$
\frac{\partial f(x)}{\partial x_{k}}=\frac{\partial}{\partial x_{k}} \sum_{i=1}^{n} b_{i} x_{i}=b_{k}
$$
由此我们可以很容易地看出$\nabla_{x} b^{T} x=b$。 这应该与单变量微积分中的类似情况进行比较，其中$\partial /(\partial x) a x=a$。
现在考虑$A\in \mathbb{S}^{n}$的二次函数$f(x)=x^TAx$。 记住这一点：
$$
f(x)=\sum_{i=1}^{n} \sum_{j=1}^{n} A_{i j} x_{i} x_{j}
$$
为了取偏导数，我们将分别考虑包括$x_k$和$x_2^k$因子的项：

$$
\begin{aligned} \frac{\partial f(x)}{\partial x_{k}} &=\frac{\partial}{\partial x_{k}} \sum_{i=1}^{n} \sum_{j=1}^{n} A_{i j} x_{i} x_{j} \\ &=\frac{\partial}{\partial x_{k}}\left[\sum_{i \neq k} \sum_{j \neq k} A_{i j} x_{i} x_{j}+\sum_{i \neq k} A_{i k} x_{i} x_{k}+\sum_{j \neq k} A_{k j} x_{k} x_{j}+A_{k k} x_{k}^{2}\right] \\ &=\sum_{i \neq k} A_{i k} x_{i}+\sum_{j \neq k} A_{k j} x_{j}+2 A_{k k} x_{k} \\ &=\sum_{i=1}^{n} A_{i k} x_{i}+\sum_{j=1}^{n} A_{k j} x_{j}=2 \sum_{i=1}^{n} A_{k i} x_{i} \end{aligned}
$$
最后一个等式，是因为$A$是对称的（我们可以安全地假设，因为它以二次形式出现）。 注意，$\nabla_{x} f(x)$的第$k$个元素是$A$和$x$的第$k$行的内积。 因此，$\nabla_{x} x^{T} A x=2 A x$。 同样，这应该提醒你单变量微积分中的类似事实，即$\partial /(\partial x) a x^{2}=2 a x$。

最后，让我们来看看二次函数$f(x)=x^TAx$黑塞矩阵（显然，线性函数$b^Tx$的黑塞矩阵为零）。在这种情况下:
$$
\frac{\partial^{2} f(x)}{\partial x_{k} \partial x_{\ell}}=\frac{\partial}{\partial x_{k}}\left[\frac{\partial f(x)}{\partial x_{\ell}}\right]=\frac{\partial}{\partial x_{k}}\left[2 \sum_{i=1}^{n} A_{\ell i} x_{i}\right]=2 A_{\ell k}=2 A_{k \ell}
$$
因此，应该很清楚$\nabla_{x}^2 x^{T} A x=2 A$，这应该是完全可以理解的（同样类似于$\partial^2 /(\partial x^2) a x^{2}=2a$的单变量事实）。

简要概括起来：

-   $\nabla_{x} b^{T} x=b$ 

-   $\nabla_{x} x^{T} A x=2 A x$ (如果$A$是对称阵)

-   $\nabla_{x}^2 x^{T} A x=2 A $  (如果$A$是对称阵)

#### 4.4 最小二乘法

让我们应用上一节中得到的方程来推导最小二乘方程。假设我们得到矩阵$A\in \mathbb{R}^{m \times n}$（为了简单起见，我们假设$A$是满秩）和向量$b\in \mathbb{R}^{m}$，从而使$b \notin \mathcal{R}(A)$。在这种情况下，我们将无法找到向量$x\in \mathbb{R}^{n}$，由于$Ax = b$，因此我们想要找到一个向量$x$，使得$Ax$尽可能接近 $b$，用欧几里德范数的平方$\|A x-b\|_{2}^{2} $来衡量。

使用公式$\|x\|^{2}=x^Tx$，我们可以得到：

$$
\begin{aligned}\|A x-b\|_{2}^{2} &=(A x-b)^{T}(A x-b) \\ &=x^{T} A^{T} A x-2 b^{T} A x+b^{T} b \end{aligned}
$$
根据$x$的梯度，并利用上一节中推导的性质：
$$
\begin{aligned} \nabla_{x}\left(x^{T} A^{T} A x-2 b^{T} A x+b^{T} b\right) &=\nabla_{x} x^{T} A^{T} A x-\nabla_{x} 2 b^{T} A x+\nabla_{x} b^{T} b \\ &=2 A^{T} A x-2 A^{T} b \end{aligned}
$$
将最后一个表达式设置为零，然后解出$x$，得到了正规方程：
$$
x = (A^TA)^{-1}A^Tb
$$
这和我们在课堂上得到的相同。

#### 4.5 行列式的梯度

现在让我们考虑一种情况，我们找到一个函数相对于矩阵的梯度，也就是说，对于$A\in \mathbb{R}^{n \times n}$，我们要找到$\nabla_{A}|A|$。回想一下我们对行列式的讨论：
$$
|A|=\sum_{i=1}^{n}(-1)^{i+j} A_{i j}\left|A_{\backslash i, \backslash j}\right| \quad(\text { for any } j \in 1, \ldots, n)
$$
所以：
$$
\frac{\partial}{\partial A_{k \ell}}|A|=\frac{\partial}{\partial A_{k \ell}} \sum_{i=1}^{n}(-1)^{i+j} A_{i j}\left|A_{\backslash i, \backslash j}\right|=(-1)^{k+\ell}\left|A_{\backslash k,\backslash \ell}\right|=(\operatorname{adj}(A))_{\ell k}
$$
从这里可以知道，它直接从伴随矩阵的性质得出：
$$
\nabla_{A}|A|=(\operatorname{adj}(A))^{T}=|A| A^{-T}
$$
现在我们来考虑函数$f : \mathbb{S}_{++}^{n} \rightarrow \mathbb{R}$，$f(A)=\log |A|$。注意，我们必须将$f$的域限制为正定矩阵，因为这确保了$|A|>0$，因此$|A|$的对数是实数。在这种情况下，我们可以使用链式法则（没什么奇怪的，只是单变量演算中的普通链式法则）来看看：
$$
\frac{\partial \log |A|}{\partial A_{i j}}=\frac{\partial \log |A|}{\partial|A|} \frac{\partial|A|}{\partial A_{i j}}=\frac{1}{|A|} \frac{\partial|A|}{\partial A_{i j}}
$$
从这一点可以明显看出：

$$
\nabla_{A} \log |A|=\frac{1}{|A|} \nabla_{A}|A|=A^{-1}
$$
我们可以在最后一个表达式中删除转置，因为$A$是对称的。注意与单值情况的相似性，其中$\partial /(\partial x) \log x=1 / x$。

#### 4.6 特征值优化

最后，我们使用矩阵演算以直接导致特征值/特征向量分析的方式求解优化问题。 考虑以下等式约束优化问题：

$$
\max _{x \in \mathbb{R}^{n}} x^{T} A x \quad \text { subject to }\|x\|_{2}^{2}=1
$$
对于对称矩阵$A\in \mathbb{S}^{n}$。求解等式约束优化问题的标准方法是采用**拉格朗日**形式，一种包含等式约束的目标函数，在这种情况下，拉格朗日函数可由以下公式给出：

$$
\mathcal{L}(x, \lambda)=x^{T} A x-\lambda x^{T} x
$$
其中，$\lambda $被称为与等式约束关联的拉格朗日乘子。可以确定，要使$x^*$成为问题的最佳点，拉格朗日的梯度必须在$x^*$处为零（这不是唯一的条件，但它是必需的）。也就是说，
$$
\nabla_{x} \mathcal{L}(x, \lambda)=\nabla_{x}\left(x^{T} A x-\lambda x^{T} x\right)=2 A^{T} x-2 \lambda x=0
$$
请注意，这只是线性方程$Ax =\lambda x$。 这表明假设$x^T x = 1$，可能最大化（或最小化）$x^T Ax$的唯一点是$A$的特征向量。

**线性代数和概率论都已经翻译完毕，请关注[github](https://github.com/fengdu78/Data-Science-Notes/tree/master/0.math)的更新，若有修改将在github上更新**

欢迎大家提交PR，对语言进行润色。

翻译：[黄海广](https://github.com/fengdu78)



================================================
FILE: 0.math/1.CS229/markdown/2.CS229-Prob.md
================================================
> 本文是斯坦福大学CS229机器学习课程的基础材料，[原始文件下载](http://cs229.stanford.edu/summer2019/cs229-prob.pdf)

> 原文作者：Arian Maleki ， Tom Do
>
> 翻译：[石振宇](https://github.com/szy2120109)
>
> 审核和修改制作：[黄海广](https://github.com/fengdu78)
>
> 备注：请关注[github](https://github.com/fengdu78/Data-Science-Notes/tree/master/0.math)的更新。



# CS229 机器学习课程复习材料-概率论

[TOC]
## 概率论复习和参考

概率论是对不确定性的研究。通过这门课，我们将依靠概率论中的概念来推导机器学习算法。这篇笔记试图涵盖适用于**CS229**的概率论基础。概率论的数学理论非常复杂，并且涉及到“分析”的一个分支：测度论。在这篇笔记中，我们提供了概率的一些基本处理方法，但是不会涉及到这些更复杂的细节。

###  1. 概率的基本要素

为了定义集合上的概率，我们需要一些基本元素，
-  样本空间$\Omega $：随机实验的所有结果的集合。在这里，每个结果 $w \in \Omega $ 可以被认为是实验结束时现实世界状态的完整描述。

- 事件集（事件空间）$\mathcal{F}$：元素 $A \in \mathcal{F}$ 的集合（称为事件）是 $\Omega $ 的子集（即每个 $A \subseteq \Omega$ 是一个实验可能结果的集合）。

  备注：$\mathcal{F}$需要满足以下三个条件：

  (1) $\emptyset  \in \mathcal{F}$

  (2) $A \in \mathcal{F} \Longrightarrow \Omega \backslash A \in \mathcal{F}$

  (3) $A_1,A_2,\cdots A_{i} \in \mathcal{F}\Longrightarrow\cup_{i} A_{i} \in \mathcal{F}$

-  概率度量$P$：函数$P$是一个$  \mathcal{F} \rightarrow \mathbb{R}$的映射，满足以下性质：
  - 对于每个 $A \in \mathcal{F}$，$P(A) \geq 0$, 
  
  - $P(\Omega) = 1$
  
  - 如果$A_1 ,A_2 ,\cdots$ 是互不相交的事件 (即 当$ i \neq j$时，$A_{i} \cap A_{j}=\emptyset$ ), 那么：
$$
P\left(\cup_{i} A_{i}\right)=\sum_{i} P\left(A_{i}\right)
$$

以上三条性质被称为**概率公理**。

**举例**：

考虑投掷六面骰子的事件。样本空间为$\Omega= \{1，2，3，4，5，6\}$。最简单的事件空间是平凡事件空间$\mathcal{F} = \{\emptyset,\Omega\}$.另一个事件空间是$\Omega$的所有子集的集合。对于第一个事件空间，满足上述要求的唯一概率度量由$P(\emptyset) = 0$，$p(\Omega)= 1$给出。对于第二个事件空间，一个有效的概率度量是将事件空间中每个事件的概率分配为$i/6$，这里$i$ 是这个事件集合中元素的数量；例如$P(\{1,2,3,4\}) =4/6$，$P(\{1,2,3\}) =3/6$。

**性质：**

- 如果$A \subseteq B$，则：$ P(A) \leq P(B)$
- $P(A \cap B) \leq min(P(A),P(B) )$
- (布尔不等式)：$P(A \cup B) \leq P(A)+P(B)$
- $P(\Omega |A ) =1-P(A)$
- (全概率定律)：如果$A_1，\cdots，A_k$是一些互不相交的事件并且它们的并集是$\Omega$，那么它们的概率之和是1

#### 1.1 条件概率和独立性

假设$B$是一个概率非0的事件，我们定义在给定$B$的条件下$A$ 的条件概率为：
$$
P(A | B) \triangleq \frac{P(A \cap B)}{P(B)}
$$
换句话说，$P(A|B$)是度量已经观测到$B$事件发生的情况下$A$事件发生的概率，两个事件被称为独立事件当且仅当$P(A \cap B) = P(A)P(B)$（或等价地，$P(A|B) = P(A)$)。因此，独立性相当于是说观察到事件$B$对于事件$A$的概率没有任何影响。

### 2. 随机变量
考虑一个实验，我们翻转10枚硬币，我们想知道正面硬币的数量。这里，样本空间$\Omega$的元素是长度为10的序列。例如，我们可能有$w_0 = \{H，H，T，H，T，H，H，T，T，T\}\in\Omega$。然而，在实践中，我们通常不关心获得任何特定正反序列的概率。相反，我们通常关心结果的实值函数，比如我们10次投掷中出现的正面数，或者最长的背面长度。在某些技术条件下，这些函数被称为**随机变量**。

更正式地说，随机变量$X$是一个的$\Omega \longrightarrow \mathbb{R}$函数。通常，我们将使用大写字母$X(\omega)$或更简单的$X$(其中隐含对随机结果$\omega$的依赖)来表示随机变量。我们将使用小写字母$x$来表示随机变量的值。

**举例：**
在我们上面的实验中，假设$X(\omega)$是在投掷序列$\omega$中出现的正面的数量。假设投掷的硬币只有10枚，那么$X(\omega)$只能取有限数量的值，因此它被称为**离散随机变量**。这里，与随机变量$X$相关联的集合取某个特定值$k$的概率为：
$$
P(X=k) :=P(\{\omega : X(\omega) =k\})
$$
**举例：**
假设$X(\omega)$是一个随机变量，表示放射性粒子衰变所需的时间。在这种情况下，$X(\omega)$具有无限多的可能值，因此它被称为**连续随机变量**。我们将$X$在两个实常数$a$和$b$之间取值的概率(其中$a < b$)表示为：
$$
P(a \leq X \leq b) :=P(\{\omega : a \leq X(\omega) \leq b\})
$$

#### 2.1 累积分布函数
为了指定处理随机变量时使用的概率度量，通常可以方便地指定替代函数(**CDF**、**PDF**和**PMF**)，在本节和接下来的两节中，我们将依次描述这些类型的函数。

**累积分布函数(CDF)**是函数$F_{X} : \mathbb{R} \rightarrow[0,1]$，它将概率度量指定为：
$$
F_{X}(x) \triangleq P(X \leq x)
$$
通过使用这个函数，我们可以计算任意事件发生的概率。图1显示了一个样本**CDF**函数。

![](images/c89fd93c5d6dccce89762d57fcd66dac.png)

<center>图1：一个累计分布函数(CDF)</center>
**性质：**

- $0 \leq F_{X}(x)\leq 1$
- $\lim _{x \rightarrow-\infty} F_{X}(x)=0$
- $\lim _{x \rightarrow\infty} F_{X}(x)=1$
- $x \leq y \Longrightarrow  F_{X}(x)\leq F_{X}(y)$

#### 2.2 概率质量函数
当随机变量$X$取有限种可能值(即，$X$是离散随机变量)时，表示与随机变量相关联的概率度量的更简单的方法是直接指定随机变量可以假设的每个值的概率。特别地，概率质量函数(**PMF**)是函数 $p_{X} : \Omega \rightarrow \mathbb{R}$，这样：
$$
p_{X}(x) \triangleq P(X=x)
$$

在离散随机变量的情况下，我们使用符号$Val(X)$表示随机变量$X$可能假设的一组可能值。例如，如果$X(\omega)$是一个随机变量，表示十次投掷硬币中的正面数，那么$Val(X) =\{0，1，2，...，10\}$。

**性质：**

- $0 \leq p_{X}(x)\leq 1$
- $\sum_{x \in V \text { al }(X)} p_{X}(x)=1$
- $\sum_{x \in A} p_{X}(x)=P(X \in A)$

#### 2.3 概率密度函数
对于一些连续随机变量，累积分布函数$F_X (x)$处可微。在这些情况下，我们将**概率密度函数(PDF)**定义为累积分布函数的导数，即：
$$
f_{X}(x) \triangleq \frac{d F_{X}(x)}{d x}
$$

请注意，连续随机变量的概率密度函数可能并不总是存在的(即，如果它不是处处可微)。 

根据微分的性质，对于很小的$\Delta x$，
$$
P(x \leq X \leq x+\Delta x) \approx f_{X}(x) \Delta x
$$
**CDF**和**PDF(**当它们存在时！)都可用于计算不同事件的概率。但是应该强调的是，任意给定点的**概率密度函数(PDF)**的值不是该事件的概率，即$f _X (x) \not = P(X = x)$。例如，$f _X (x)$可以取大于1的值(但是$f _X (x)$在$\mathbb{R}$的任何子集上的积分最多为1)。

**性质：**

- $f_X(x)\geq 0$
- $\int_{-\infty}^{\infty} f_{X}(x)=1$
- $\int_{x \in A} f_{X}(x) d x=P(X \in A)$

#### 2.4 期望
假设$X$是一个离散随机变量，其**PMF**为 $p_X (x)$，$g : \mathbb{R} \longrightarrow \mathbb{R}$是一个任意函数。在这种情况下，$g(X)$可以被视为随机变量，我们将$g(X)$的期望值定义为：
$$
E[g(X)] \triangleq \sum_{x \in V a l(X)} g(x) p_{X}(x)
$$
如果$X$是一个连续的随机变量，其**PDF** 为$f _X (x)$，那么$g(X)$的期望值被定义为：
$$
E[g(X)] \triangleq \int_{-\infty}^{\infty} g(x) f_{X}(x) d x
$$

直觉上，$g(X)$的期望值可以被认为是$g(x)$对于不同的$x$值可以取的值的“加权平均值”，其中权重由$p_X(x)$或$f_X(x)$给出。作为上述情况的特例，请注意，随机变量本身的期望值，是通过令$g(x) = x$得到的，这也被称为随机变量的平均值。

**性质：**

- 对于任意常数 $a \in \mathbb{R}$，$E[a]=a$
- 对于任意常数 $a \in \mathbb{R}$，$E[af(X)]=aE[f(X)]$
- (线性期望)：$E[f(X)+g(X)]=E[f(X)]+E[g(X)]$
- 对于一个离散随机变量$X$，$E[1\{X=k\}]=P(X=k)$

#### 2.5 方差
随机变量$X$的**方差**是随机变量$X$的分布围绕其平均值集中程度的度量。形式上，随机变量$X$的方差定义为：
$$
\operatorname{Var}[X] \triangleq E\left[(X-E(X))^{2}\right]
$$
使用上一节中的性质，我们可以导出方差的替代表达式:
$$
\begin{aligned} E\left[(X-E[X])^{2}\right] &=E\left[X^{2}-2 E[X] X+E[X]^{2}\right] \\ &=E\left[X^{2}\right]-2 E[X] E[X]+E[X]^{2} \\ &=E\left[X^{2}\right]-E[X]^{2} \end{aligned}
$$

其中第二个等式来自期望的线性，以及$E[X]$相对于外层期望实际上是常数的事实。

**性质：**

- 对于任意常数 $a \in \mathbb{R}$，$Val[a]=0$
- 对于任意常数 $a \in \mathbb{R}$，$Var[af(X)]=a^2Var[f(X)]$

**举例：**

计算均匀随机变量$X$的平均值和方差，任意$x \in [0，1]$，其**PDF**为 $p_X(x)= 1$，其他地方为0。
$$
E[X]=\int_{-\infty}^{\infty} x f_{X}(x) d x=\int_{0}^{1} x d x=\frac{1}{2}
$$
$$
E\left[X^{2}\right]=\int_{-\infty}^{\infty} x^{2} f_{X}(x) d x=\int_{0}^{1} x^{2} d x=\frac{1}{3}
$$
$$
Var[X]=E[X^2]-E[X]^2=\frac{1}{3}-\frac{1}{4}=\frac{1}{12}
$$
**举例：**

假设对于一些子集$A \subseteq \Omega$，有$g(x) = 1\{x \in A\}$，计算$E[g(X)]$?

**离散情况：**
$$
E[g(X)]=\sum_{x \in V a l(X)} 1\{x \in A\} P_{X}(x) d x=\sum_{x \in A} P_{X}(x) d x=P(x \in A)
$$

**连续情况：**
$$
E[g(X)]=\int_{-\infty}^{\infty} 1\{x \in A\} f_{X}(x) d x=\int_{x \in A} f_{X}(x) d x=P(x \in A)
$$

#### 2.6 一些常见的随机变量
**离散随机变量**

- 伯努利分布：硬币掷出正面的概率为$p$（其中：$0 \leq p \leq 1$），如果正面发生，则为1，否则为0。
  $$
  p(x)=\left\{\begin{array}{ll}{p} & {\text { if } p=1} \\ {1-p} & {\text { if } p=0}\end{array}\right.
  $$

- 二项式分布：掷出正面概率为$p$（其中：$0 \leq p \leq 1$）的硬币$n$次独立投掷中正面的数量。
$$
p(x)=\left(\begin{array}{l}{n} \\ {x}\end{array}\right) p^{x}(1-p)^{n-x}
$$

- 几何分布：掷出正面概率为$p$（其中：$p >0$）的硬币第一次掷出正面所需要的次数。

  

- 泊松分布：用于模拟罕见事件频率的非负整数的概率分布（其中：$\lambda >0$）。
$$
p(x)=e^{-\lambda} \frac{\lambda^{x}}{x !}
$$

**连续随机变量**

- 均匀分布：在$a$和$b$之间每个点概率密度相等的分布（其中：$a<b$）。
$$
f(x)=\left\{\begin{array}{ll}{\frac{1}{b-a}} & {\text { if } a \leq x \leq b} \\ {0} & {\text { otherwise }}\end{array}\right.
$$

- 指数分布：在非负实数上有衰减的概率密度（其中：$\lambda >0$）。
$$
f(x)=\left\{\begin{array}{ll}{\lambda e^{-\lambda x}} & {\text { if } x \geq 0} \\ {0} & {\text { otherwise }}\end{array}\right.
$$

- 正态分布：又被称为高斯分布。
$$
f(x)=\frac{1}{\sqrt{2 \pi} \sigma} e^{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}}
$$


一些随机变量的概率密度函数和累积分布函数的形状如图2所示。

![](images/b958c16cfdce9e6bd2b810b10d71416e.png)

<center>图2：一些随机变量的概率密度函数(PDF)和累积分布函数(CDF)</center>
下表总结了这些分布的一些特性：

|                分布                |          概率密度函数(PDF)或者概率质量函数(**PMF**)          |        均值         |         方差          |
| :--------------------------------: | :----------------------------------------------------------: | :-----------------: | :-------------------: |
|     $Bernoulli(p)$(伯努利分布)     | $\left\{\begin{array}{ll}{p} & {\text { if } x=1} \\ {1-p} & {\text { if } x=0}\end{array}\right.$ |         $p$         |       $p(1-p)$        |
|    $Binomial(n,p)$(二项式分布)     | $\left(\begin{array}{l}{n} \\ {k}\end{array}\right) p^{k}(1-p)^{n-k}$  其中：$0 \leq k \leq n$ |        $np$         |         $npq$         |
|      $Geometric(p)$(几何分布)      |             $p(1-p)^{k-1}$  其中：$k=1,2,\cdots$             |    $\frac{1}{p}$    |  $\frac {1-p}{p^2}$   |
|    $Poisson(\lambda)$(泊松分布)    |    $e^{-\lambda} \lambda^{x} / x !$  其中：$k=1,2,\cdots$    |      $\lambda$      |       $\lambda$       |
|      $Uniform(a,b)$(均匀分布)      |              $\frac{1}{b-a}$ 存在$x \in (a,b)$               |   $\frac{a+b}{2}$   | $\frac{(b-a)^2}{12}$  |
| $Gaussian(\mu,\sigma^2)$(高斯分布) | $\frac{1}{\sqrt{2 \pi} \sigma} e^{-\frac{1}{2 \sigma^{2}}(x-\mu)^{2}}$ |        $\mu$        |      $\sigma^2$       |
|  $Exponential(\lambda)$(指数分布)  |        $\lambda e^{-\lambda x}$   $x\geq0,\lambda>0$         | $\frac{1}{\lambda}$ | $\frac{1}{\lambda^2}$ |

### 3. 两个随机变量
到目前为止，我们已经考虑了单个随机变量。然而，在许多情况下，在随机实验中，我们可能有不止一个感兴趣的量。例如，在一个我们掷硬币十次的实验中，我们可能既关心$X(\omega) =$出现的正面数量，也关心$Y (\omega) =$连续最长出现正面的长度。在本节中，我们考虑两个随机变量的设置。

#### 3.1 联合分布和边缘分布
假设我们有两个随机变量，一个方法是分别考虑它们。如果我们这样做，我们只需要$F_X (x)$和$F_Y (y)$。但是如果我们想知道在随机实验的结果中，$X$和$Y$同时假设的值，我们需要一个更复杂的结构，称为$X$和$Y$的**联合累积分布函数**，定义如下:
$$
F_{XY}(x,y)=P(X \leq x,Y \leq y)
$$


可以证明，通过了解联合累积分布函数，可以计算出任何涉及到$X$和$Y$的事件的概率。

联合**CDF**: $F_{XY }(x,y)$和每个变量的联合分布函数$F_X(x)$和$F_Y (y)$分别由下式关联:
$$
F_{X}(x)=\lim _{y \rightarrow \infty} F_{X Y}(x, y) d y
$$

$$
F_{Y}(y)=\lim _{y \rightarrow \infty} F_{X Y}(x, y) dx
$$
这里我们称$F_X(x)$和$F_Y (y)$为 $F_{XY }(x,y)$的**边缘累积概率分布函数**。

**性质：**

- $0 \leq F_{XY }(x,y) \leq 1$
- $\lim _{x, y \rightarrow \infty} F_{X Y}(x, y)=1$
- $\lim _{x, y \rightarrow -\infty} F_{X Y}(x, y)=0$
- $F_{X}(x)=\lim _{y \rightarrow \infty} F_{X Y}(x, y)$

#### 3.2 联合概率和边缘概率质量函数
如果$X$和$Y$是离散随机变量，那么**联合概率质量函数**  $p_{X Y} : \mathbb{R} \times \mathbb{R} \rightarrow [0,1]$由下式定义：

$$
p_{X Y}(x,y)=P(X=x,Y=y)
$$

这里, 对于任意$x$，$y$，$0 \leq P_{XY} (x,y) \leq 1$, 并且 $\sum_{x \in V a l(X)} \sum_{y \in V a l(Y)} P_{X Y}(x, y)=1$

两个变量上的**联合PMF**分别与每个变量的概率质量函数有什么关系？事实上：
$$
p_{X}(x)=\sum_{y} p_{X Y}(x, y)
$$

对于$p_Y (y)$类似。在这种情况下，我们称$p_X(x)$为$X$的边际概率质量函数。在统计学中，将一个变量相加形成另一个变量的边缘分布的过程通常称为“边缘化”。

#### 3.3 联合概率和边缘概率密度函数
假设$X$和$Y$是两个连续的随机变量，具有联合分布函数$F_{XY}$。在$F_{XY}(x,y)$在$x$和$y$中处处可微的情况下，我们可以定义**联合概率密度函数**：
$$
f_{X Y}(x, y)=\frac{\partial^{2} F_{X Y}(x, y)}{\partial x \partial y}
$$
如同在一维情况下，$f_{XY}(x,y)\not= P(X = x,Y = y)$，而是：
$$
\iint_{x \in A} f_{X Y}(x, y) d x d y=P((X, Y) \in A)
$$

请注意，概率密度函数$f_{XY}(x,y)$的值总是非负的，但它们可能大于1。尽管如此，可以肯定的是 $\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f_{X Y}(x, y)=1$

与离散情况相似，我们定义:
$$
f_{X}(x)=\int_{-\infty}^{\infty} f_{X Y}(x, y) d y
$$
作为$X$的**边际概率密度函数**(或**边际密度**)，对于$f_Y (y)$也类似。

#### 3.4 条件概率分布
条件分布试图回答这样一个问题，当我们知道$X$必须取某个值$x$时，$Y$上的概率分布是什么？在离散情况下，给定$Y$的条件概率质量函数是简单的：
$$
p_{Y | X}(y | x)=\frac{p_{X Y}(x, y)}{p_{X}(x)}
$$
假设分母不等于0。

在连续的情况下，在技术上要复杂一点，因为连续随机变量的概率等于零。忽略这一技术点，我们通过类比离散情况，简单地定义给定$X = x$的条件概率密度为：
$$
f_{Y | X}(y | x)=\frac{f_{X Y}(x, y)}{f_{X}(x)}
$$
假设分母不等于0。

#### 3.5 贝叶斯定理
当试图推导一个变量给定另一个变量的条件概率表达式时，经常出现的一个有用公式是**贝叶斯定理**。

对于离散随机变量$X$和$Y$：
$$
P_{Y | X}(y | x)=\frac{{P_{XY}}(x, y)}{P_{X}(x)}=\frac{P_{X | Y}(x | y) P_{Y}(y)}{\sum_{y^{\prime} \in V a l(Y)} P_{X | Y}\left(x | y^{\prime}\right) P_{Y}\left(y^{\prime}\right)}
$$

对于连续随机变量$X$和$Y$：

$$
f_{Y | X}(y | x)=\frac{f_{X Y}(x, y)}{f_{X}(x)}=\frac{f_{X | Y}(x | y) f_{Y}(y)}{\int_{-\infty}^{\infty} f_{X | Y}\left(x | y^{\prime}\right) f_{Y}\left(y^{\prime}\right) d y^{\prime}}
$$

#### 3.6 独立性
如果对于$X$和$Y$的所有值，$F_{XY}(x,y) = F_X(x)F_Y(y)$，则两个随机变量$X$和$Y$是独立的。等价地，

- 对于离散随机变量, 对于任意$x \in Val(X)$, $y \in Val(Y)$ ，$p_{XY}(x,y) = p_X (x)p_Y (y)$。
- 对于离散随机变量, $p_Y |X (y|x) = p_Y (y)$当对于任意$y \in Val(Y)$且$p_X (x) \not= 0$。
- 对于连续随机变量, $f_{XY}(x,y) = f_X (x)f_Y(y)$ 对于任意 $x,y \in \mathbb{R}$。
- 对于连续随机变量, $f_{Y |X} (y|x) = f_Y (y)$ ，当$f_X (x)\not = 0$对于任意$y \in \mathbb{R}$。

非正式地说，如果“知道”一个变量的值永远不会对另一个变量的条件概率分布有任何影响，那么两个随机变量$X$和$Y$是独立的，也就是说，你只要知道$f(x)$和$f(y)$就知道关于这对变量$(X，Y)$的所有信息。以下引理将这一观察形式化:

**引理3.1** 

如果$X$和$Y$是独立的，那么对于任何$A，B⊆ \mathbb{R}$，我们有：
$$
P(X \in A, Y \in B)=P(X \in A) P(Y \in B)
$$
利用上述引理，我们可以证明如果$X$与$Y$无关，那么$X$的任何函数都与$Y$的任何函数无关。



#### 3.7 期望和协方差
假设我们有两个离散的随机变量$X$，$Y$并且$g : \mathbf{R}^{2} \longrightarrow \mathbf{R}$是这两个随机变量的函数。那么$g$的期望值以如下方式定义：
$$
E[g(X, Y)] \triangleq \sum_{x \in V a l(X)} \sum_{y \in V a l(Y)} g(x, y) p_{X Y}(x, y)
$$
对于连续随机变量$X$，$Y$，类似的表达式是：
$$
E[g(X, Y)]=\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(x, y) f_{X Y}(x, y) d x d y
$$
我们可以用期望的概念来研究两个随机变量之间的关系。特别地，两个随机变量的**协方差**定义为：
$$
{Cov}[X, Y] \triangleq E[(X-E[X])(Y-E[Y])]
$$
使用类似于方差的推导，我们可以将它重写为：
$$
\begin{aligned} {Cov}[X, Y] &=E[(X-E[X])(Y-E[Y])] \\ &=E[X Y-X E[Y]-Y E[X]+E[X] E[Y]] \\ &=E[X Y]-E[X] E[Y]-E[Y] E[X]+E[X] E[Y]] \\ &=E[X Y]-E[X] E[Y] \end{aligned}
$$

在这里，说明两种协方差形式相等的关键步骤是第三个等号，在这里我们使用了这样一个事实，即$E[X]$和$E[Y]$实际上是常数，可以被提出来。当$cov[X，Y] = 0$时，我们说$X$和$Y$不相关。

**性质：**

- (期望线性) $E[f(X,Y ) + g(X,Y)] = E[f(X,Y )] + E[g(X,Y)]$
- $V ar[X + Y ] = V ar[X] + V ar[Y ] + 2Cov[X,Y]$
- 如果$X$和$Y$相互独立, 那么 $Cov[X,Y ] = 0$
- 如果$X$和$Y$相互独立, 那么 $E[f(X)g(Y )] = E[f(X)]E[g(Y)]$.


### 4. 多个随机变量
上一节介绍的概念和想法可以推广到两个以上的随机变量。特别是，假设我们有$n$个连续随机变量，$X _1 (\omega),X_2 (\omega),\cdots X_n (\omega)$。在本节中，为了表示简单，我们只关注连续的情况，对离散随机变量的推广工作类似。

#### 4.1 基本性质
我们可以定义$X_1,X_2,\cdots,X_n$的**联合累积分布函数**、**联合概率密度函数**，以及给定$X_2,\cdots,X_n$时$X_1$的**边缘概率密度函数**为：
$$
F_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots x_{n}\right)=P\left(X_{1} \leq x_{1}, X_{2} \leq x_{2}, \ldots, X_{n} \leq x_{n}\right)
$$

$$
f_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots x_{n}\right)=\frac{\partial^{n} F_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots x_{n}\right)}{\partial x_{1} \ldots \partial x_{n}}
$$
$$
f_{X_{1}}\left(X_{1}\right)=\int_{-\infty}^{\infty} \cdots \int_{-\infty}^{\infty} f_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots x_{n}\right) d x_{2} \ldots d x_{n}
$$

$$
f_{X_{1} | X_{2}, \ldots, X_{n}}\left(x_{1} | x_{2}, \dots x_{n}\right)=\frac{f_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \dots x_{n}\right)}{f_{X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots x_{n}\right)}
$$

为了计算事件$A \subseteq \mathbb{R}^{n}$的概率，我们有：
$$
P\left(\left(x_{1}, x_{2}, \ldots x_{n}\right) \in A\right)=\int_{\left(x_{1}, x_{2}, \ldots x_{n}\right) \in A} f_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots x_{n}\right) d x_{1} d x_{2} \ldots d x_{n}
$$
**链式法则：**

从多个随机变量的条件概率的定义中，可以看出：
$$
\begin{aligned} f\left(x_{1}, x_{2}, \ldots, x_{n}\right) &=f\left(x_{n} | x_{1}, x_{2} \ldots, x_{n-1}\right) f\left(x_{1}, x_{2} \ldots, x_{n-1}\right) \\ &=f\left(x_{n} | x_{1}, x_{2} \ldots, x_{n-1}\right) f\left(x_{n-1} | x_{1}, x_{2} \ldots, x_{n-2}\right) f\left(x_{1}, x_{2} \ldots, x_{n-2}\right) \\ &=\cdots=f\left(x_{1}\right) \prod_{i=2}^{n} f\left(x_{i} | x_{1}, \ldots, x_{i-1}\right) \end{aligned}
$$

独立性:对于多个事件，$A_1,\cdots ,A_k$,我们说$A_1,\cdots ,A_k$ 是相互独立的,当对于任何子集$S \subseteq \{1，2,\cdots,k\}$，我们有：
$$
P\left(\cap_{i \in S} A_{i}\right)=\prod_{i \in S} P\left(A_{i}\right)
$$

同样，我们说随机变量$X_1,X_2,\cdots,X_n$是独立的，如果：

$$
f(x_1,\cdots,x_n)=f(x_1)f(x_2)\cdots f(x_n)
$$
这里，相互独立性的定义只是两个随机变量独立性到多个随机变量的自然推广。 

独立随机变量经常出现在机器学习算法中，其中我们假设属于训练集的训练样本代表来自某个未知概率分布的独立样本。为了明确独立性的重要性，考虑一个“坏的”训练集，我们首先从某个未知分布中抽取一个训练样本$(x^{ (1)},y^{(1)})$，然后将完全相同的训练样本的$m-1$个副本添加到训练集中。在这种情况下，我们有：
$$
P\left(\left(x^{(1)}, y^{(1)}\right), \ldots .\left(x^{(m)}, y^{(m)}\right)\right) \neq \prod_{i=1}^{m} P\left(x^{(i)}, y^{(i)}\right)
$$

尽管训练集的大小为$m$，但这些例子并不独立！虽然这里描述的过程显然不是为机器学习算法建立训练集的明智方法，但是事实证明，在实践中，样本的不独立性确实经常出现，并且它具有减小训练集的“有效大小”的效果。

#### 4.2 随机向量
假设我们有n个随机变量。当把所有这些随机变量放在一起工作时，我们经常会发现把它们放在一个向量中是很方便的...我们称结果向量为随机向量(更正式地说，随机向量是从$\Omega$到$\mathbb{R}^n$的映射)。应该清楚的是，随机向量只是处理$n$个随机变量的一种替代符号，因此联合概率密度函数和综合密度函数的概念也将适用于随机向量。

**期望:**

考虑$g : \mathbb{R}^n \rightarrow \mathbb{R}$中的任意函数。这个函数的期望值 被定义为
$$
E[g(X)]=\int_{\mathbb{R}^{n}} g\left(x_{1}, x_{2}, \ldots, x_{n}\right) f_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots x_{n}\right) d x_{1} d x_{2} \ldots d x_{n}E[g(X)]\\=\int_{\mathbb{R}^{n}} g\left(x_{1}, x_{2}, \ldots, x_{n}\right) f_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots x_{n}\right) d x_{1} d x_{2} \ldots d x_{n}
$$
其中，$\int_{\mathbb{R}^{n}}$是从$-\infty$到$\infty$的$n$个连续积分。如果$g$是从$\mathbb{R}^n$到$\mathbb{R}^m$的函数，那么$g$的期望值是输出向量的元素期望值，即，如果$g$是：




$$
g(x)=\left[\begin{array}{c}{g_{1}(x)} \\ {g_{2}(x)} \\ {\vdots} \\ {g_{m}(x)}\end{array}\right]
$$
那么，
$$
E[g(X)]=\left[\begin{array}{c}{E\left[g_{1}(X)\right]} \\ {E\left[g_{2}(X)\right]} \\ {\vdots} \\ {E\left[g_{m}(X)\right]}\end{array}\right]
$$



协方差矩阵：对于给定的随机向量$X:\Omega\rightarrow \mathbb{R}^n$，其协方差矩阵$\Sigma$是$n \times n$平方矩阵，其输入由$\Sigma_{i j}={Cov}\left[X_{i}, X_{j}\right]$给出。从协方差的定义来看，我们有：
$$
\begin{aligned}
\begin{equation}
\Sigma=\left[\begin{array}{ccc}{{Cov}\left[X_{1}, X_{1}\right]} & {\cdots} & {{Cov}\left[X_{1}, X_{n}\right]} \\ {\vdots} & {\ddots} & {\vdots} \\ {{Cov}\left[X_{n}, X_{1}\right]} & {\cdots} & {{Cov}\left[X_{n}, X_{n}\right]}\end{array}\right]\\

=\left[\begin{array}{ccc}{E\left[X_{1}^{2}\right]-E\left[X_{1}\right] E\left[X_{1}\right]} & {\cdots} & {E\left[X_{1} X_{n}\right]-E\left[X_{1}\right] E\left[X_{n}\right]} \\ {\vdots} & {\ddots} & {\vdots} \\ {E\left[X_{n} X_{1}\right]-E\left[X_{n}\right] E\left[X_{1}\right]} & {\cdots} & {E\left[X_{n}^{2}\right]-E\left[X_{n}\right] E\left[X_{n}\right]}\end{array}\right]\\

=\left[\begin{array}{ccc}{E\left[X_{1}^{2}\right]} & {\cdots} & {E\left[X_{1} X_{n}\right]} \\ {\vdots} & {\ddots} & {\vdots} \\ {E\left[X_{n} X_{1}\right]} & {\cdots} & {E\left[X_{n}^{2}\right]}\end{array}\right]-\left[\begin{array}{ccc}{E\left[X_{1}\right] E\left[X_{1}\right]} & {\cdots} & {E\left[X_{1}\right] E\left[X_{n}\right]} \\ {\vdots} & {\ddots} & {\vdots} \\ {E\left[X_{n}\right] E\left[X_{1}\right]} & {\cdots} & {E\left[X_{n}\right] E\left[X_{n}\right]}\end{array}\right]\\
=E\left[X X^{T}\right]-E[X] E[X]^{T}=\ldots=E\left[(X-E[X])(X-E[X])^{T}\right]

\end{equation}
\end{aligned}
$$
其中矩阵期望以明显的方式定义。 
协方差矩阵有许多有用的属性:

- $\Sigma \succeq 0$；也就是说，$\Sigma$是正半定的。 
- $\Sigma=\Sigma^T$；也就是说，$\Sigma$是对称的。

#### 4.3 多元高斯分布
随机向量上概率分布的一个特别重要的例子叫做多元高斯或多元正态分布。随机向量$X\in \mathbb{R}^n$被认为具有多元正态(或高斯)分布，当其具有均值$\mu \in \mathbb{R}^n$和协方差矩阵$\Sigma \in \mathbb{S}_{++}^{n}$(其中$ \mathbb{S}_{++}^{n}$指对称正定$n \times n$矩阵的空间)

$f_{X_{1}, X_{2}, \ldots, X_{n}}\left(x_{1}, x_{2}, \ldots, x_{n} ; \mu, \Sigma\right)=\frac{1}{(2 \pi)^{n / 2}|\Sigma|^{1 / 2}} \exp \left(-\frac{1}{2}(x-\mu)^{T} \Sigma^{-1}(x-\mu)\right)$

我们把它写成$X \sim \mathcal{N}(\mu, \Sigma)$。请注意，在$n = 1$的情况下，它降维成普通正态分布，其中均值参数为$\mu_1$，方差为$\Sigma_{11}$。 

一般来说，高斯随机变量在机器学习和统计中非常有用，主要有两个原因：

首先，在统计算法中对“噪声”建模时，它们非常常见。通常，噪声可以被认为是影响测量过程的大量小的独立随机扰动的累积；根据中心极限定理，独立随机变量的总和将趋向于“看起来像高斯”。

其次，高斯随机变量便于许多分析操作，因为实际中出现的许多涉及高斯分布的积分都有简单的封闭形式解。我们将在本课程稍后遇到这种情况。

### 5. 其他资源
一本关于**CS229**所需概率水平的好教科书是谢尔顿·罗斯的《概率第一课》(*A First Course on Probability* by Sheldon Ross)。




================================================
FILE: 0.math/1.CS229/markdown/README.md
================================================
本文是斯坦福大学CS 229机器学习课程的基础材料，[原始文件下载](http://cs229.stanford.edu/summer2019/cs229-linalg.pdf)

翻译：[黄海广](https://github.com/fengdu78)
备注：请关注[github](https://github.com/fengdu78/Data-Science-Notes/tree/master/0.math)的更新，近期将更新完。

markdown文件夹是文件的markdown代码，内容与pdf一致，是为了方便研究者写论文或者文章使用。

github内容是方便用户学习研究使用，请勿用于商业目的。



================================================
FILE: 0.math/README.md
================================================
# 机器学习的数学基础材料

## 一、机器学习的数学基础

[0.basic](0.basic)，这个是从大学教材里搜集的机器学习数学基础资料，分为高等数学，线性代数、概率论与数理统计三部分。主要内容是以前考研考博时候的数学笔记，难度应该在本科3年级左右。

**数据科学需要一定的数学基础，但仅仅做应用的话，如果时间不多，不用学太深，了解基本公式即可，遇到问题再查吧。**



## 二、斯坦福大学CS229机器学习课程复习材料

[1.CS229](1.CS229)文件夹的材料斯坦福大学CS 229机器学习课程的基础材料的翻译版本，[原始文件下载](http://cs229.stanford.edu/summer2019/cs229-linalg.pdf)

翻译：[黄海广](https://github.com/fengdu78)
备注：请关注[github](https://github.com/fengdu78/Data-Science-Notes/tree/master/0.math)的更新，近期将更新完。

文件介绍：

[1.CS229线性代数（翻译完毕）](1.CS229-LinearAlgebra.pdf)

github内容是方便用户学习研究使用，请勿用于商业目的。

如果需要引用这个Repo:

格式： `fengdu78, Data-Science-Notes, (2019), GitHub repository, https://github.com/fengdu78/Data-Science-Notes`

================================================
FILE: 1.python-basic/Python-100/README.md
================================================
# Python练习题100题-带你轻松入门Python
> 近日发现一个Python入门的练习仓库，作者收集100多道Python的常见练习题，几乎概括了Python初学要掌握的基本问题。

如果您是python的初学者，那么这100多个练习可以帮助您轻松地使用Python。仓库给定的问题非常简单和容易理解。初学者每天可以尝试3-5个问题，需要一点时间来解决，但肯定会学到一些新的东西（不管你有多懒），经过一个月的常规练习，把练习做完看懂，只要能独立解决100多个问题，基本上Python就入门了。练习题作者地址： https://github.com/darkprinx/100-plus-Python-programming-exercises-extended

## 仓库内容

这个仓库一共有24个Jupyter notebook文件（每天一个），每个文件有3-5个练习，一共104题。问题模板格式

- 提问
- 提示
- 解答

在这个仓库是用Python3 语言。作者修正一些随机错误和变量命名，符合PEP8规范，仓库分为Jupyter notebook格式（文件夹名称:notebooks）和Markdown格式（文件夹名称：Status）两个文件夹，两者内容一致。


================================================
FILE: 1.python-basic/Python-100/Status/Day 1.md
================================================
# Question 1

### **Question:**

> **_Write a program which will find all such numbers which are divisible by 7 but are not a multiple of 5,
> between 2000 and 3200 (both included).The numbers obtained should be printed in a comma-separated sequence on a single line._**

---

### Hints:

> **_Consider use range(#begin, #end) method._**

---

**Main author's Solution: Python 2**

```python
l=[]
for i in range(2000, 3201):
    if (i%7==0) and (i%5!=0):
        l.append(str(i))

print ','.join(l)
```

---

**My Solution: Python 3**
- **Using for loops**

```python
for i in range(2000,3201):
    if i%7 == 0 and i%5!=0:
        print(i,end=',')
print("\b")
```

---
- **Using generators and list comprehension**

```python
print(*(i for i in range(2000, 3201) if i%7 == 0 and i%5 != 0), sep=",")
```
# Question 2

### **Question:**

> **_Write a program which can compute the factorial of a given numbers.The results should be printed in a comma-separated sequence on a single line.Suppose the following input is supplied to the program: 8
> Then, the output should be:40320_**

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
def fact(x):
    if x == 0:
        return 1
    return x * fact(x - 1)

x = int(raw_input())
print fact(x)
```

---

**My Solution: Python 3**

- **Using While Loop**
  ```python
  n = int(input()) #input() function takes input as string type
                   #int() converts it to integer type
  fact = 1
  i = 1
  while i <= n:
      fact = fact * i;
      i = i + 1
  print(fact)
  ```
- **Using For Loop**
  ```python
  n = int(input()) #input() function takes input as string type
                  #int() converts it to integer type
  fact = 1
  for i in range(1,n+1):
      fact = fact * i
  print(fact)
  ```
- **Using Lambda Function**

  ```python
  # Solution by:  harshraj22

  n = int(input())
  def shortFact(x): return 1 if x <= 1 else x*shortFact(x-1)
  print(shortFact(n))

  ```
---
```python
'''Solution by: minnielahoti
'''

while True:
try:
    num = int(input("Enter a number: "))
    break
except ValueError as err:
    print(err)

org = num
fact = 1
while num:
    fact = num * fact
    num = num - 1
print(f'the factorial of {org} is {fact}')
```
---
```python
'''Soltuion by: KruthikaSR
'''
from functools import reduce

def fun(acc, item):
	return acc*item

num = int(input())
print(reduce(fun,range(1, num+1), 1))
```
---

# Question 3

### **Question:**

> **_With a given integral number n, write a program to generate a dictionary that contains (i, i x i) such that is an integral number between 1 and n (both included). and then the program should print the dictionary.Suppose the following input is supplied to the program: 8_**

> **_Then, the output should be:_**

```
{1: 1, 2: 4, 3: 9, 4: 16, 5: 25, 6: 36, 7: 49, 8: 64}
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input.Consider use dict()_**

---

**Main author's Solution: Python 2**

```python
n = int(raw_input())
d = dict()
for i in range(1,n+1):
    d[i] = i * i
print d
```

**My Solution: Python 3:**

- **Using for loop**

```python
n = int(input())
ans = {}
for i in range (1,n+1):
    ans[i] = i * i
print(ans)
```

- **Using dictionary comprehension**

```python
n = int(input())
ans={i : i*i for i in range(1,n+1)}
print(ans)
```
---
```python
'''Solution by: minnielahoti
   Corrected by: TheNobleKnight 
'''

try:
    num = int(input("Enter a number: "))
except ValueError as err:
    print(err)

dictio = dict()
for item in range(num+1):
    if item == 0:
        continue
    else:
	dictio[item] = item * item
print(dictio)
```
---
```python
'''Solution by: yurbika
'''

num = int(input("Number: "))
print(dict(list(enumerate((i * i for i in range(num+1))))))
```
---
## Conclusion

**_These was the solved problems of day 1. The above problems are very easy for the basic syntex learners.I have shown some easy ways of coding in my solutions. Lets see how to face and attack new problems in the next day._**

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%202.md "Next Day")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day 2.md
================================================
# Question 4

### **Question:**

> **_Write a program which accepts a sequence of comma-separated numbers from console and generate a list and a tuple which contains every number.Suppose the following input is supplied to the program:_**

```
34,67,55,33,12,98
```

> **_Then, the output should be:_**

```
['34', '67', '55', '33', '12', '98']
('34', '67', '55', '33', '12', '98')
```

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input.tuple() method can convert list to tuple_**

---

**Main author's Solution: Python 2**

```python
values = raw_input()
l = values.split(",")
t = tuple(l)
print l
print t
```

---

**My Solution: Python 3**

```python
lst = input().split(',')  # the input is being taken as string and as it is string it has a built in
                          # method name split. ',' inside split function does split where it finds any ','
                          # and save the input as list in lst variable

tpl = tuple(lst)          # tuple method converts list to tuple

print(lst)
print(tpl)
```
---
```python 
'''solution by: minnielahoti
'''
print(tuple(input("Enter a series of numbers separated by a comma :").split(',')))
```
---

# Question 5

### **Question:**

> **_Define a class which has at least two methods:_**
>
> - **_getString: to get a string from console input_**
> - **_printString: to print the string in upper case._**

> **_Also please include simple test function to test the class methods._**

### Hints:

> **_Use **init** method to construct some parameters_**

---

**Main author's Solution: Python 2**

```python
class InputOutString(object):
    def __init__(self):
        self.s = ""

    def get_string(self):
        self.s = raw_input()

    def print_string(self):
        print self.s.upper()

str_obj = InputOutString()
str_obj.get_string()
str_obj.print_string()
```

---

**My Solution: Python 3**

```python
class IOstring():
    def get_string(self):
        self.s = input()

    def print_string(self):
        print(self.s.upper())

xx = IOstring()
xx.get_string()
xx.print_string()
```

---

# Question 6

### **Question:**

> **_Write a program that calculates and prints the value according to the given formula:_**

> **_Q = Square root of [(2 _ C _ D)/H]_**

> **_Following are the fixed values of C and H:_**

> **_C is 50. H is 30._**

> **_D is the variable whose values should be input to your program in a comma-separated sequence.For example
> Let us assume the following comma separated input sequence is given to the program:_**

```
100,150,180
```

> **_The output of the program should be:_**

```
18,22,24
```

---

### Hints:

> **_If the output received is in decimal form, it should be rounded off to its nearest value (for example, if the output received is 26.0, it should be printed as 26).In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
import math
c = 50
h = 30
value = []
items = [x for x in raw_input().split(',')]
for d in items:
    value.append(str(int(round(math.sqrt(2*c*float(d)/h)))))

print ','.join(value)
```

---

**My Solution: Python 3**

```python
from math import sqrt # import specific functions as importing all using *
                      # is bad practice

C,H = 50,30

def calc(D):
    return sqrt((2*C*D)/H)

D = [int(i) for i in input().split(',')] # splits in comma position and set up in list
D = [int(i) for i in D]   # converts string to integer
D = [calc(i) for i in D]  # returns floating value by calc method for every item in D
D = [round(i) for i in D] # All the floating values are rounded
D = [str(i) for i in D]   # All the integers are converted to string to be able to apply join operation

print(",".join(D))
```

**OR**

```python
from math import sqrt

C,H = 50,30

def calc(D):
    return sqrt((2*C*D)/H)

D = input().split(',')                     # splits in comma position and set up in list
D = [str(round(calc(int(i)))) for i in D]  # using comprehension method. It works in order of the previous code
print(",".join(D))
```

**OR**

```python
from math import sqrt
C,H = 50,30

def calc(D):
    return sqrt((2*C*D)/H)

print(",".join([str(int(calc(int(i)))) for i in input().split(',')]))
```

**OR**

```python
from math import * # importing all math functions
C,H = 50,30

def calc(D):
    D = int(D)
    return str(int(sqrt((2*C*D)/H)))

D = input().split(',')
D = list(map(calc,D))   # applying calc function on D and storing as a list
print(",".join(D))
```
---
```python
'''Solution by: parian5
'''
from math import sqrt
C, H = 50, 30
mylist = input().split(',')
print(*(round(sqrt(2*C*int(D)/H)) for D in mylist), sep=",")
```
---

# Question 7

### **Question:**

> **_Write a program which takes 2 digits, X,Y as input and generates a 2-dimensional array. The element value in the i-th row and j-th column of the array should be i _ j.\***

> **_Note: i=0,1.., X-1; j=0,1,¡­Y-1. Suppose the following inputs are given to the program: 3,5_**

> **_Then, the output of the program should be:_**

```
[[0, 0, 0, 0, 0], [0, 1, 2, 3, 4], [0, 2, 4, 6, 8]]
```

---

### Hints:

> **_Note: In case of input data being supplied to the question, it should be assumed to be a console input in a comma-separated form._**

---

**Main author's Solution: Python 2**

```python
input_str = raw_input()
dimensions = [int(x) for x in input_str.split(',')]
row_num = dimensions[0]
col_num = dimensions[1]
multilist = [[0 for col in range(col_num)] for row in range(row_num)]

for row in range(row_num):
    for col in range(col_num):
        multilist[row][col] = row * col

print multilist
```

---

**My Solution: Python 3**

```python
x,y = map(int,input().split(','))
lst = []

for i in range(x):
    tmp = []
    for j in range(y):
        tmp.append(i*j)
    lst.append(tmp)

print(lst)
```

**OR**

```python
x,y = map(int,input().split(','))
lst = [[i*j for j in range(y)] for i in range(x)]
print(lst)
```

---

# Question 8

### **Question:**

> **_Write a program that accepts a comma separated sequence of words as input and prints the words in a comma-separated sequence after sorting them alphabetically._**

> **_Suppose the following input is supplied to the program:_**

```
without,hello,bag,world
```

> **_Then, the output should be:_**

```
bag,hello,without,world
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
items = [x for x in raw_input().split(',')]
items.sort()
print ','.join(items)
```

---

**My Solution: Python 3**

```python
lst = input().split(',')
lst.sort()
print(",".join(lst))
```

---

# Question 9

### **Question:**

> **_Write a program that accepts sequence of lines as input and prints the lines after making all characters in the sentence capitalized._**

> **_Suppose the following input is supplied to the program:_**

```
Hello world
Practice makes perfect
```

> **_Then, the output should be:_**

```
HELLO WORLD
PRACTICE MAKES PERFECT
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
lines = []
while True:
    s = raw_input()
    if s:
        lines.append(s.upper())
    else:
        break

for sentence in lines:
    print sentence
```

---

**My Solution: Python 3**

```python
lst = []

while True:
    x = input()
    if len(x)==0:
        break
    lst.append(x.upper())

for line in lst:
    print(line)
```

**OR**

```python
def user_input():
    while True:
        s = input()
        if not s:
            return
        yield s

for line in map(str.upper, user_input()):
    print(line)
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%201.md "Day 1")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%203.md "Day 3")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day 3.md
================================================
# Question 10

### **Question**

> **_Write a program that accepts a sequence of whitespace separated words as input and prints the words after removing all duplicate words and sorting them alphanumerically._**

> **_Suppose the following input is supplied to the program:_**

```
hello world and practice makes perfect and hello world again
```

> **_Then, the output should be:_**

```
again and hello makes perfect practice world
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input.We use set container to remove duplicated data automatically and then use sorted() to sort the data._**

---

**Main author's Solution: Python 2**

```python
s = raw_input()
words = [word for word in s.split(" ")]
print " ".join(sorted(list(set(words))))
```

---

**My Solution: Python 3**

```python
word = input().split()

for i in word:
    if word.count(i) > 1:    #count function returns total repeatation of an element that is send as argument
        word.remove(i)     # removes exactly one element per call

word.sort()
print(" ".join(word))
```

**OR**

```python
word = input().split()
[word.remove(i) for i in word if word.count(i) > 1 ]   # removal operation with comprehension method
word.sort()
print(" ".join(word))
```

**OR**

```python
word = sorted(list(set(input().split())))              #  input string splits -> converting into set() to store unique
                                                       #  element -> converting into list to be able to apply sort
print(" ".join(word))
```

---
```python
'''Solution by: Sukanya-Mahapatra
'''
inp_string = input("Enter string: ").split()
out_string = []
for words in inp_string:
    if words not in out_string:
        out_string.append(words)
print(" ".join(sorted(out_string)))
```
---

# Question 11

### **Question**

> **_Write a program which accepts a sequence of comma separated 4 digit binary numbers as its input and then check whether they are divisible by 5 or not. The numbers that are divisible by 5 are to be printed in a comma separated sequence._**

> **_Example:_**

```
0100,0011,1010,1001
```

> **_Then the output should be:_**

```
1010
```

> **_Notes: Assume the data is input by console._**

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
value = []
items=[x for x in raw_input().split(',')]
for p in items:
    intp = int(p,2)
    if not intp % 5:
        value.append(p)

print ','.join(value)
```

---

**My Solution: Python 3**

```python
def check(x):                       # converts binary to integer & returns zero if divisible by 5
    total,pw = 0,1
    reversed(x)

    for i in x:
        total+=pw * (ord(i) - 48)   # ord() function returns ASCII value
        pw*=2
    return total % 5

data = input().split(",")           # inputs taken here and splited in ',' position
lst = []

for i in data:
    if check(i) == 0:               # if zero found it means divisible by zero and added to the list
        lst.append(i)

print(",".join(lst))
```

**OR**

```python
def check(x):                   # check function returns true if divisible by 5
    return int(x,2)%5 == 0      # int(x,b) takes x as string and b as base from which
                                # it will be converted to decimal
data = input().split(',')

data = list(filter(check,data)) # in filter(func,object) function, elements are picked from 'data' if found True by 'check' function
print(",".join(data))
```

**OR**

```python
data = input().split(',')
data = list(filter(lambda i:int(i,2)%5==0,data))    # lambda is an operator that helps to write function of one line
print(",".join(data))
```

---

```python
'''Solution by: nikitaMogilev
'''
data = input().split(',')
data = [num for num in data if int(num, 2) % 5 == 0]
print(','.join(data))
```

---

# Question 12

### **Question:**

> **_Write a program, which will find all such numbers between 1000 and 3000 (both included) such that each digit of the number is an even number.The numbers obtained should be printed in a comma-separated sequence on a single line._**

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
values = []
for i in range(1000, 3001):
    s = str(i)
    if (int(s[0])%2 == 0) and (int(s[1])%2 == 0) and (int(s[2])%2 == 0) and (int(s[3])%2 == 0):
        values.append(s)
print ",".join(values)
```

---

**My Solution: Python 3**

```python
lst = []

for i in range(1000,3001):
    flag = 1
    for j in str(i):          # every integer number i is converted into string
        if ord(j)%2 != 0:     # ord returns ASCII value and j is every digit of i
            flag = 0          # flag becomes zero if any odd digit found
    if flag == 1:
        lst.append(str(i))    # i is stored in list as string

print(",".join(lst))
```

**OR**

```python
def check(element):
    return all(ord(i)%2 == 0 for i in element)  # all returns True if all digits i is even in element

lst = [str(i) for i in range(1000,3001)]        # creates list of all given numbers with string data type
lst = list(filter(check,lst))                   # filter removes element from list if check condition fails
print(",".join(lst))
```

**OR**

```python
lst = [str(i) for i in range(1000,3001)]
lst = list(filter(lambda i:all(ord(j)%2 == 0 for j in i), lst))   # using lambda to define function inside filter function
print(",".join(lst))
```

---

```python
'''Solution by: nikitaMogilev
'''
# map() digits of each number with lambda function and check if all() of them even
# str(num) gives us opportunity to iterate through number by map() and join()
print(','.join([str(num) for num in range(1000, 3001) if all(map(lambda num: int(num) % 2 == 0, str(num)))]))
```

---

# Question 13

### **Question:**

> **_Write a program that accepts a sentence and calculate the number of letters and digits._**

> **_Suppose the following input is supplied to the program:_**

```
hello world! 123
```

> **_Then, the output should be:_**

```
LETTERS 10
DIGITS 3
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
s = raw_input()
d = {"DIGITS":0, "LETTERS":0}
for c in s:
    if c.isdigit():
        d["DIGITS"]+=1
    elif c.isalpha():
        d["LETTERS"]+=1
    else:
        pass
print "LETTERS", d["LETTERS"]
print "DIGITS", d["DIGITS"]
```

---

**My Solution: Python 3**

```python
word = input()
letter,digit = 0,0

for i in word:
    if ('a'<=i and i<='z') or ('A'<=i and i<='Z'):
        letter+=1
    if '0'<=i and i<='9':
        digit+=1

print("LETTERS {0}\nDIGITS {1}".format(letter,digit))
```

**OR**

```python
word = input()
letter, digit = 0,0

for i in word:
    if i.isalpha(): # returns True if alphabet
        letter += 1
    elif i.isnumeric(): # returns True if numeric
        digit += 1
print(f"LETTERS {letter}\n{digits}") # two different types of formating method is shown in both solution
```
---
```python
''' Solution by: popomaticbubble
'''
import re

input_string = input('> ')
print()
counter = {"LETTERS":len(re.findall("[a-zA-Z]", input_string)), "NUMBERS":len(re.findall("[0-9]", input_string))}

print(counter)
```
---

## Conclusion

**_All the above problems are mostly string related problems. Major parts of the solution includes string releted functions and comprehension method to write down the code in more shorter form._**

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%202.md "Day 2")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%204.md "Day 4")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day 4.md
================================================
# Question 14

### **Question:**

> **_Write a program that accepts a sentence and calculate the number of upper case letters and lower case letters._**

> **_Suppose the following input is supplied to the program:_**

```
Hello world!
```

> **_Then, the output should be:_**

```
UPPER CASE 1
LOWER CASE 9
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
s = raw_input()
d = {"UPPER CASE":0, "LOWER CASE":0}
for c in s:
    if c.isupper():
        d["UPPER CASE"]+=1
    elif c.islower():
        d["LOWER CASE"]+=1
    else:
        pass
print "UPPER CASE", d["UPPER CASE"]
print "LOWER CASE", d["LOWER CASE"]
```

---

**My Solution: Python 3**

```python
word = input()
upper,lower = 0,0

for i in word:
    if 'a'<=i and i<='z' :
        lower+=1
    if 'A'<=i and i<='Z':
        upper+=1

print("UPPER CASE {0}\nLOWER CASE {1}".format(upper,lower))
```

**OR**

```python
word = input()
upper,lower = 0,0

for i in word:
        lower+=i.islower()
        upper+=i.isupper()

print("UPPER CASE {0}\nLOWER CASE {1}".format(upper,lower))
```

**OR**

```python
word = input()
upper = sum(1 for i in word if i.isupper())           # sum function cumulatively sum up 1's if the condition is True
lower = sum(1 for i in word if i.islower())

print("UPPER CASE {0}\nLOWER CASE {1}".format(upper,lower))
```

**OR**

```python
# solution by Amitewu

string = input("Enter the sentense")
upper = 0
lower = 0
for x in string:
    if x.isupper() == True:
        upper += 1
    if x.islower() == True:
        lower += 1

print("UPPER CASE: ", upper)
print("LOWER CASE: ", lower)
```

---

# Question 15

### **Question:**

> **_Write a program that computes the value of a+aa+aaa+aaaa with a given digit as the value of a._**

> **_Suppose the following input is supplied to the program:_**

```
9
```

> **_Then, the output should be:_**

```
11106
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
a = raw_input()
n1 = int( "%s" % a )
n2 = int( "%s%s" % (a,a) )
n3 = int( "%s%s%s" % (a,a,a) )
n4 = int( "%s%s%s%s" % (a,a,a,a) )
print n1+n2+n3+n4
```

---

**My Solution: Python 3**

```python
a = input()
total,tmp = 0,str()        # initialing an integer and empty string

for i in range(4):
    tmp+=a               # concatenating 'a' to 'tmp'
    total+=int(tmp)      # converting string type to integer type

print(total)
```

**OR**

```python
a = input()
total = int(a) + int(2*a) + int(3*a) + int(4*a)  # N*a=Na, for example  a="23", 2*a="2323",3*a="232323"
print(total)
```
---
```python
'''Solution by: ChichiLovesDonkeys
'''
from functools import reduce
x = input('please enter a digit:')
reduce(lambda x, y: int(x) + int(y), [x*i for i in range(1,5)])
```
---
```python
'''Solution by: lcastrooliveira
'''
def question_15(string_digit):
    return sum(int(string_digit * n) for n in range(1, 5))

inp = input()
print(question_15(inp))
```
---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%203.md "Day 3")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%205.md "Day 5")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day 5.md
================================================
# Question 16

### **Question:**

> **_Use a list comprehension to square each odd number in a list. The list is input by a sequence of comma-separated numbers._** >**_Suppose the following input is supplied to the program:_**

```
1,2,3,4,5,6,7,8,9
```

> **_Then, the output should be:_**

```
1,9,25,49,81
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
## The solution by the author is incorrect.Thus it's not included here.
```

---

**My Solution: Python 3**

```python
lst = [str(int(i)**2) for i in input().split(',') if int(i) % 2]
print(",".join(lst))
```

---

```python
'''Solution by: shagun'''
square odd no

lst = input().split(',')     # splits in comma position and set up in list

seq = []
lst = [int(i) for i in lst]  # converts string to integer
for i in lst:
        if i%2 != 0:
                i = i*i
                seq.append(i)


seq = [str(i) for i in seq]   # All the integers are converted to string to be able to apply join operation
print(",".join(seq))
```

---

**_There were a mistake in the the test case and the solution's whice were notified and fixed with the help of @dwedigital. My warm thanks to him._**

# Question 17

### **Question:**

> **_Write a program that computes the net amount of a bank account based a transaction log from console input. The transaction log format is shown as following:_**

```
D 100
W 200
```

- D means deposit while W means withdrawal.

> **_Suppose the following input is supplied to the program:_**

```
D 300
D 300
W 200
D 100
```

> **_Then, the output should be:_**

```
500
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
import sys
netAmount = 0
while True:
    s = raw_input()
    if not s:
        break
    values = s.split(" ")
    operation = values[0]
    amount = int(values[1])
    if operation=="D":
        netAmount+=amount
    elif operation=="W":
        netAmount-=amount
    else:
        pass
print netAmount
```

---

**My Solution: Python 3**

```python
total = 0
while True:
    s = input().split()
    if not s:            # break if the string is empty
        break
    cm,num = map(str,s)    # two inputs are distributed in cm and num in string data type

    if cm=='D':
        total+=int(num)
    if cm=='W':
        total-=int(num)

print(total)
```

---

```python
'''Solution by: leonedott'''

lst = []
while True:
  x = input()
  if len(x)==0:
    break
  lst.append(x)

balance = 0
for item in lst:
  if 'D' in item:
    balance += int(item.strip('D '))
  if 'W' in item:
    balance -= int(item.strip('W '))
print(balance)
```

---

```python
'''Solution by: AlexanderSro'''

account = 0
while True:
    action = input("Deposit/Whitdrow/Balance/Quit? D/W/B/Q: ").lower()
    if action == "d":
        deposit = input("How much would you like to deposit? ")
        account = account + int(deposit)
    elif action == "w":
        withdrow = input("How much would you like to withdrow? ")
        account = account - int(withdrow)
    elif action == "b":
        print(account)
    else:
        quit()
```

---

```python
'''Solution by: ShalomPrinz
'''
lines = []
while True:
	loopInput = input()
	if loopInput == "done":
		break
	else:
		lines.append(loopInput)

lst = list(int(i[2:]) if i[0] == 'D' else -int(i[2:]) for i in lines)
print(sum(lst))
```
---
```python
'''Solution by: popomaticbubble 
'''
transactions = []

while True:
    text = input("> ")
    if text:
    	text = text.strip('D ')
    	text = text.replace('W ', '-')
    	transactions.append(text)
    else: 
		break	
		
transactions = (int(i) for i in transactions)
balance = sum(transactions)
print(f"Balance is {balance}")
```
---
```python
'''Solution by: ChichiLovesDonkeys
'''

money = 0
while 1:
    trans = input().split(' ')
    if trans[0] == 'D':
        money = money + int(trans[1])
    elif trans[0] == 'W':
        money = money - int(trans[1])
    elif input() == '':
        break
    print(f'Your current balance is: {money}')
```
---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%204.md "Day 4")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%206.md "Day 6")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day 6.md
================================================
# Question 18

### **Question:**

> **_A website requires the users to input username and password to register. Write a program to check the validity of password input by users._**

> **_Following are the criteria for checking the password:_**

- **_At least 1 letter between [a-z]_**
- **_At least 1 number between [0-9]_**
- **_At least 1 letter between [A-Z]_**
- **_At least 1 character from [$#@]_**
- **_Minimum length of transaction password: 6_**
- **_Maximum length of transaction password: 12_**

> **_Your program should accept a sequence of comma separated passwords and will check them according to the above criteria. Passwords that match the criteria are to be printed, each separated by a comma._**

> **_Example_**

> **_If the following passwords are given as input to the program:_**

```
ABd1234@1,a F1#,2w3E*,2We3345
```

> **_Then, the output of the program should be:_**

```
ABd1234@1
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
import re
value = []
items = [x for x in raw_input().split(',')]
for p in items:
    if len(p) < 6 or len(p) > 12:
        continue
    else:
        pass
    if not re.search("[a-z]",p):
        continue
    elif not re.search("[0-9]",p):
        continue
    elif not re.search("[A-Z]",p):
        continue
    elif not re.search("[$#@]",p):
        continue
    elif re.search("\s",p):
        continue
    else:
        pass
    value.append(p)
print ",".join(value)
```

---

**My Solution: Python 3**

```python
def is_low(x):                  # Returns True  if the string has a lowercase
    for i in x:
        if 'a'<=i and i<='z':
            return True
    return  False

def is_up(x):                   # Returns True  if the string has a uppercase
    for i in x:
        if 'A'<= i and i<='Z':
            return True
    return  False

def is_num(x):                  # Returns True  if the string has a numeric digit
    for i in x:
        if '0'<=i and i<='9':
            return True
    return  False

def is_other(x):                # Returns True if the string has any "$#@"
    for i in x:
        if i=='$' or i=='#' or i=='@':
            return True
    return False

s = input().split(',')
lst = []

for i in s:
    length = len(i)
    if 6 <= length and length <= 12 and is_low(i) and is_up(i) and is_num(i) and is_other(i):   #Checks if all the requirments are fulfilled
        lst.append(i)

print(",".join(lst))
```

**OR**

```python
def check(x):
    cnt = (6<=len(x) and len(x)<=12)
    for i in x:
        if i.isupper():
            cnt+=1
            break
    for i in x:
        if i.islower():
            cnt+=1
            break
    for i in x:
        if i.isnumeric():
            cnt+=1
            break
    for i in x:
        if i=='@' or i=='#'or i=='$':
            cnt+=1
            break
    return cnt == 5               # counting if total 5 all conditions are fulfilled then returns True

s = input().split(',')
lst = filter(check,s)             # Filter function pick the words from s, those returns True by check() function
print(",".join(lst))
```

**OR**

```python
import  re

s = input().split(',')
lst = []

for i in s:
    cnt = 0
    cnt+=(6<=len(i) and len(i)<=12)
    cnt+=bool(re.search("[a-z]",i))      # here re module includes a function re.search() which returns the object information
    cnt+=bool(re.search("[A-Z]",i))      # of where the pattern string i is matched with any of the [a-z]/[A-z]/[0=9]/[@#$] characters
    cnt+=bool(re.search("[0-9]",i))      # if not a single match found then returns NONE which converts to False in boolean
    cnt+=bool(re.search("[@#$]",i))      # expression otherwise True if found any.
    if cnt == 5:
        lst.append(i)

print(",".join(lst))
```

---

```python
'''Solution by: pratikb0501
'''
import re
a = input('Enter passwords: ').split(',')
pass_pattern = re.compile(r"^(?=.*[0-9])(?=.*[a-z])(?=.*[A-Z])(?=.*[$#@]).{6,12}$")
for i in a:
    if pass_pattern.fullmatch(i):
        print(i)
```

---

# Question 19

### **Question:**

> **_You are required to write a program to sort the (name, age, score) tuples by ascending order where name is string, age and score are numbers. The tuples are input by console. The sort criteria is:_**

- **_1: Sort based on name_**
- **_2: Then sort based on age_**
- **_3: Then sort by score_**

> **_The priority is that name > age > score._**

> **_If the following tuples are given as input to the program:_**

```
Tom,19,80
John,20,90
Jony,17,91
Jony,17,93
Json,21,85
```

> **_Then, the output of the program should be:_**

```
[('John', '20', '90'), ('Jony', '17', '91'), ('Jony', '17', '93'), ('Json', '21', '85'), ('Tom', '19', '80')]
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input.We use itemgetter to enable multiple sort keys._**

---

**Main author's Solution: Python 2**

```python
from operator import itemgetter, attrgetter

l = []
while True:
    s = raw_input()
    if not s:
        break
    l.append(tuple(s.split(",")))

print sorted(l, key=itemgetter(0,1,2))
```

---

**My Solution: Python 3**

```python
lst = []
while True:
    s = input().split(',')
    if not s[0]:                          # breaks for blank input
        break
    lst.append(tuple(s))

lst.sort(key= lambda x:(x[0],x[1],x[2]))  # here key is defined by lambda and the data is sorted by element priority 0>1>2 in accending order
print(lst)
```

---

## Conclusion

**_Before the above problems, I didn't even know about re(regular expression) module and its use. I didn't even know how to sort by multiple keys. To solve those problems in different ways I had to explore and learn those syntax.There are a lots of interesting stuffs in re module though I faced quite a bit hardship to understand many of them._**

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%205.md "Day 5")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%207.md "Day 7")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day 7.md
================================================
# Question 20

### **Question:**

> **_Define a class with a generator which can iterate the numbers, which are divisible by 7, between a given range 0 and n._**

---

### Hints:

> **_Consider use class, function and comprehension._**

---

**Main author's Solution: Python 2**

#### **_The solution code for this problem was not as reltive to as the problem mentioned and there was a typing mistake while calling the function._**

---

**Solution: Python 3**

```python
'''Solution by: ShalomPrinz
'''
class MyGen():
    def by_seven(self, n):
        for i in range(0, int(n/7) + 1):
            yield i * 7

for i in MyGen().by_seven( int(input('Please enter a number... ')) ):
    print(i)
```

---

```python
'''Solution by: Seawolf159
'''
class Divisible:

    def by_seven(self, n):
        for number in range(1,n + 1):
            if number % 7 == 0: yield number


divisible = Divisible()
generator = divisible.by_seven(int(input("Please insert a number. --> ")))
for number in generator:
    print(number)

```

---

# Question 21

### **Question:**

> **_A robot moves in a plane starting from the original point (0,0). The robot can move toward UP, DOWN, LEFT and RIGHT with a given steps. The trace of robot movement is shown as the following:_**

```
UP 5
DOWN 3
LEFT 3
RIGHT 2
```

> **_The numbers after the direction are steps. Please write a program to compute the distance from current position after a sequence of movement and original point. If the distance is a float, then just print the nearest integer._**
> **_Example:_**
> **_If the following tuples are given as input to the program:_**

```
UP 5
DOWN 3
LEFT 3
RIGHT 2
```

> **_Then, the output of the program should be:_**

```
2
```

---

### Hints:

> **_In case of input data being supplied to the question, it should be assumed to be a console input.Here distance indicates to euclidean distance.Import math module to use sqrt function._**

---

**Main author's Solution: Python 2**

```python
import math
pos = [0,0]
while True:
    s = raw_input()
    if not s:
        break
    movement = s.split(" ")
    direction = movement[0]
    steps = int(movement[1])
    if direction=="UP":
        pos[0]+=steps
    elif direction=="DOWN":
        pos[0]-=steps
    elif direction=="LEFT":
        pos[1]-=steps
    elif direction=="RIGHT":
        pos[1]+=steps
    else:
        pass

print int(round(math.sqrt(pos[1]**2+pos[0]**2)))
```

---

**My Solution: Python 3**

```python
import  math

x,y = 0,0
while True:
    s = input().split()
    if not s:
        break
    if s[0]=='UP':                  # s[0] indicates command
        x-=int(s[1])                # s[1] indicates unit of move
    if s[0]=='DOWN':
        x+=int(s[1])
    if s[0]=='LEFT':
        y-=int(s[1])
    if s[0]=='RIGHT':
        y+=int(s[1])
                                    # N**P means N^P
dist = round(math.sqrt(x**2 + y**2))  # euclidean distance = square root of (x^2+y^2) and rounding it to nearest integer
print(dist)
```
---
```python
'''Solution by: pratikb0501
'''

from math import sqrt
lst = []
position = [0,0]
while True:
    a = input()
    if not a:
        break
    lst.append(a)
for i in lst:
    if 'UP' in i:
        position[0] -= int(i.strip('UP '))
    if 'DOWN' in i:
        position[0] += int(i.strip('DOWN '))
    if 'LEFT' in i:
        position[1] -= int(i.strip('LEFT '))
    if 'RIGHT' in i:
        position[1] += int(i.strip('RIGHT '))
print(round(sqrt(position[1] ** 2 + position[0] ** 2)))
```
---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%206.md "Day 6")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%208.md "Day 8")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day 8.md
================================================
# Question 22

### **Question:**

> **_Write a program to compute the frequency of the words from the input. The output should output after sorting the key alphanumerically._**

> **_Suppose the following input is supplied to the program:_**

```
New to Python or choosing between Python 2 and Python 3? Read Python 2 or Python 3.
```

> **_Then, the output should be:_**

```
2:2
3.:1
3?:1
New:1
Python:5
Read:1
and:1
between:1
choosing:1
or:2
to:1
```

---

### Hints

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

**Main author's Solution: Python 2**

```python
freq = {}   # frequency of words in text
line = raw_input()
for word in line.split():
    freq[word] = freq.get(word,0)+1

words = freq.keys()
words.sort()

for w in words:
    print "%s:%d" % (w,freq[w])
```

---

**My Solution: Python 3**

```python
ss = input().split()
word = sorted(set(ss))     # split words are stored and sorted as a set

for i in word:
    print("{0}:{1}".format(i,ss.count(i)))
```

**OR**

```python
ss = input().split()
dict = {}
for i in ss:
    i = dict.setdefault(i,ss.count(i))    # setdefault() function takes key & value to set it as dictionary.

dict = sorted(dict.items())               # items() function returns both key & value of dictionary as a list
                                          # and then sorted. The sort by default occurs in order of 1st -> 2nd key
for i in dict:
    print("%s:%d"%(i[0],i[1]))
```

**OR**

```python
ss = input().split()
dict = {i:ss.count(i) for i in ss}     # sets dictionary as i-> split word & ss.count(i) -> total occurrence of i in ss
dict = sorted(dict.items())            # items() function returns both key & value of dictionary as a list
                                       # and then sorted. The sort by default occurs in order of 1st -> 2nd key
for i in dict:
    print("%s:%d"%(i[0],i[1]))
```

**OR**

```python
from collections import Counter

ss = input().split()
ss = Counter(ss)         # returns key & frequency as a dictionary
ss = sorted(ss.items())  # returns as a tuple list

for i in ss:
    print("%s:%d"%(i[0],i[1]))
```

**Solution by: AnjanKumarG**

```python
from pprint import pprint
p=input().split()
pprint({i:p.count(i) for i in p})
```

---

# Question 23

### **Question:**

> **_Write a method which can calculate square value of number_**

---

### Hints:

```
Using the ** operator which can be written as n**p where means n^p
```

---

**Main author's Solution: Python 2**

```python
def square(num):
    return num ** 2

print square(2)
print square(3)
```

---

**My Solution: Python 3**

```python
n=int(input())
print(n**2)
```

---

# Question 24

### **Question:**

> **_Python has many built-in functions, and if you do not know how to use it, you can read document online or find some books. But Python has a built-in document function for every built-in functions._**

> **_Please write a program to print some Python built-in functions documents, such as abs(), int(), raw_input()_**

> **_And add document for your own function_**

### Hints:

```
The built-in document method is __doc__
```

---

**Main author's Solution: Python 2**

```python
print abs.__doc__
print int.__doc__
print raw_input.__doc__

def square(num):
    '''Return the square value of the input number.

    The input number must be integer.
    '''
    return num ** 2

print square(2)
print square.__doc__
```

---

**My Solution: Python 3**

```python
print(str.__doc__)
print(sorted.__doc__)

def pow(n,p):
    '''
    param n: This is any integer number
    param p: This is power over n
    return:  n to the power p = n^p
    '''

    return n**p

print(pow(3,4))
print(pow.__doc__)
```

---

# Question 25

### **Question:**

> **_Define a class, which have a class parameter and have a same instance parameter._**

---

### Hints:

```
Define an instance parameter, need add it in __init__ method.You can init an object with construct parameter or set the value later
```

---

**Main author's Solution: Python 2**

```python
class Person:
    # Define the class parameter "name"
    name = "Person"

    def __init__(self, name = None):
        # self.name is the instance parameter
        self.name = name

jeffrey = Person("Jeffrey")
print "%s name is %s" % (Person.name, jeffrey.name)

nico = Person()
nico.name = "Nico"
print "%s name is %s" % (Person.name, nico.name)
```

---

**My Solution: Python 3**

```python
class Car:
    name = "Car"

    def __init__(self,name = None):
        self.name = name

honda=Car("Honda")
print("%s name is %s"%(Car.name,honda.name))

toyota=Car()
toyota.name="Toyota"
print("%s name is %s"%(Car.name,toyota.name))
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%207.md "Day 7")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%209.md "Day 9")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day 9.md
================================================
# Question 26

### **Question:**

> **_Define a function which can compute the sum of two numbers._**

---

### Hints:

> **_Define a function with two numbers as arguments. You can compute the sum in the function and return the value._**

---

**Main author's Solution: Python 2**

```python
def SumFunction(number1, number2):
	return number1 + number2

print SumFunction(1,2)
```

---

**My Solution: Python 3**

```python
sum = lambda n1,n2 : n1 + n2      # here lambda is use to define little function as sum
print(sum(1,2))
```

---

# Question 27

### **Question:**

> **_Define a function that can convert a integer into a string and print it in console._**

---

### Hints:

> **_Use str() to convert a number to string._**

---

**Main author's Solution: Python 2**

```python
def printValue(n):
	print str(n)

printValue(3)
```

---

**My Solution: Python 3**

```python
conv = lambda x : str(x)
n = conv(10)
print(n)
print(type(n))            # checks the type of the variable
```

---

# Question 28

### **Question:**

> **_Define a function that can receive two integer numbers in string form and compute their sum and then print it in console._**

---

### Hints:

> **_Use int() to convert a string to integer._**

---

**Main author's Solution: Python 2**

```python
def printValue(s1,s2):
	print int(s1) + int(s2)
printValue("3","4") #7
```

---

**My Solution: Python 3**

```python
sum = lambda s1,s2 : int(s1) + int(s2)
print(sum("10","45"))      # 55
```

---

# Question 29

### **Question:**

> **_Define a function that can accept two strings as input and concatenate them and then print it in console._**

---

### Hints:

> **_Use + sign to concatenate the strings._**

---

**Main author's Solution: Python 2**

```python
def printValue(s1,s2):
	print s1 + s2

printValue("3","4") #34
```

---

**My Solution: Python 3**

```python
sum = lambda s1,s2 : s1 + s2
print(sum("10","45"))        # 1045
```

---

# Question 30

### **Question:**

> **_Define a function that can accept two strings as input and print the string with maximum length in console. If two strings have the same length, then the function should print all strings line by line._**

---

### Hints:

> **_Use len() function to get the length of a string._**

---

**Main author's Solution: Python 2**

```python
def printValue(s1,s2):
	len1 = len(s1)
	len2 = len(s2)
	if len1 > len2:
		print s1
	elif len2 > len1:
		print s2
	else:
		print s1
		print s2

printValue("one","three")

```

---

**My Solution: Python 3**

```python
def printVal(s1,s2):
    len1 = len(s1)
    len2 = len(s2)
    if len1 > len2:
        print(s1)
    elif len1 < len2:
        print(s2)
    else:
        print(s1)
        print(s2)

s1,s2=input().split()
printVal(s1,s2)
```

---

```python
'''Solution by: yuan1z'''
func = lambda a,b: print(max((a,b),key=len)) if len(a)!=len(b) else print(a+'\n'+b)
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%208.md "Day 9")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_10.md "Day 10")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_10.md
================================================
# Question 31

### **Question:**

> **_Define a function which can print a dictionary where the keys are numbers between 1 and 20 (both included) and the values are square of keys._**

---

### Hints:

```
Use dict[key]=value pattern to put entry into a dictionary.Use ** operator to get power of a number.Use range() for loops.
```

---

**Main Author's Solution: Python 2**

```python
def printDict():
	d=dict()
	for i in range(1,21):
		d[i]=i**2
	print d

printDict()
```

---

**My Solution: Python 3**

```python
def printDict():
    dict={i:i**2 for i in range(1,21)}   # Using comprehension method and
    print(dict)

printDict()
```

---

# Question 32

### **Question:**

> **_Define a function which can generate a dictionary where the keys are numbers between 1 and 20 (both included) and the values are square of keys. The function should just print the keys only._**

---

### Hints:

```
Use dict[key]=value pattern to put entry into a dictionary.Use ** operator to get power of a number.Use range() for loops.Use keys() to iterate keys in the dictionary. Also we can use item() to get key/value pairs.
```

---

**Main Author's Solution: Python 2**

```python
def printDict():
	d=dict()
	for i in range(1,21):
		d[i]=i**2
	for k in d.keys():
		print k
printDict()
```

---

**My Solution: Python 3**

```python
def printDict():
    dict = {i: i**2 for i in range(1, 21)}
    print(dict.keys())      # print keys of a dictionary

printDict()
```

---

# Question 33

### **Question:**

> **_Define a function which can generate and print a list where the values are square of numbers between 1 and 20 (both included)._**

---

### Hints:

```
Use ** operator to get power of a number.Use range() for loops.Use list.append() to add values into a list.
```

---

**Main Author's Solution: Python 2**

```python
def printList():
	li=list()
	for i in range(1,21):
		li.append(i**2)
	print li

printList()
```

---

**My Solution: Python 3**

```python
def printList():
    lst = [i ** 2 for i in range(1, 21)]
    print(lst)

printList()
```

---

# Question 34

### **Question:**

> **_Define a function which can generate a list where the values are square of numbers between 1 and 20 (both included). Then the function needs to print the first 5 elements in the list._**

---

### Hints:

```
Use ** operator to get power of a number.Use range() for loops.Use list.append() to add values into a list.Use [n1:n2] to slice a list
```

---

**Main Author's Solution: Python 2**

```python
def printList():
	li=list()
	for i in range(1,21):
		li.append(i**2)
	print li[:5]

printList()
```

---

**My Solution: Python 3**

```python
def printList():
    lst = [i ** 2 for i in range(1, 21)]

    for i in range(5):
        print(lst[i])

printList()
```
---
```python
'''Solution by: popomaticbubble
'''
def squares(n):
    squares_list = [i**2 for i in range(1,n+1)]
    print(squares_list[0:5])
squares(20)
```
---

```python
'''Solution by: yuan1z'''
func = lambda :print([i**2 for i in range(1,21)][:5])
```

---

# Question 35

### **Question:**

> **_Define a function which can generate a list where the values are square of numbers between 1 and 20 (both included). Then the function needs to print the last 5 elements in the list._**

---

### Hints:

```
Use ** operator to get power of a number.Use range() for loops.Use list.append() to add values into a list.Use [n1:n2] to slice a list
```

---

**Main Author's Solution: Python 2**

```python
def printList():
	li=list()
	for i in range(1,21):
		li.append(i**2)
	print li[-5:]

printList()
```

---

**My Solution: Python 3**

```python
def printList():
    lst = [i ** 2 for i in range(1, 21)]
    for i in range(19,14,-1):
        print(lst[i])

printList()
```
---
```python
'''Solution by: popomaticbubble
'''
def squares(n):
    squares_list = [i**2 for i in range(1,n+1)]
    print(squares_list[-5:])
squares(20)
```

---

# Question 36

### **Question:**

> **_Define a function which can generate a list where the values are square of numbers between 1 and 20 (both included). Then the function needs to print all values except the first 5 elements in the list._**

---

```
Hints: Use ** operator to get power of a number.Use range() for loops.Use list.append() to add values into a list.Use [n1:n2] to slice a list
```

---

**Main Author's Solution: Python 2**

```python
def printList():
	li=list()
	for i in range(1,21):
		li.append(i**2)
	print li[5:]

printList()
```

---

**My Solution: Python 3**

```python
def printList():
    lst = [i ** 2 for i in range(1, 21)]
    for i in range(5,20):
        print(lst[i])

printList()
```

---

# Question 37

### **Question:**

> **_Define a function which can generate and print a tuple where the value are square of numbers between 1 and 20 (both included)._**

---

### Hints:

```
Use ** operator to get power of a number.Use range() for loops.Use list.append() to add values into a list.Use tuple() to get a tuple from a list.
```

---

**Main Author's Solution: Python 2**

```python
def printTuple():
	li=list()
	for i in range(1,21):
		li.append(i**2)
	print tuple(li)

printTuple()
```

---

**My Solution: Python 3**

```python
def printTupple():
    lst = [i ** 2 for i in range(1, 21)]
    print(tuple(lst))

printTupple()
```

---

```python
'''
Solution by: Seawolf159
'''
def square_of_numbers():
    return tuple(i ** 2 for i in range(1, 21))

print(square_of_numbers())
```

---

### Comment

**_Problems of this section is very much easy and all of those are of a modification of same type problem which mainly focused on using some commonly used function works with list,dictionary, tupple.In my entire solutions, I havn't tried to solve problems in efficient way.Rather I tried to solve in a different way that I can.This will help a beginner to know how simplest problems can be solved in different ways._**

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day%209.md "Day 9")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_11.md "Day 11")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_11.md
================================================
# Question 38

### **Question:**

> **_With a given tuple (1,2,3,4,5,6,7,8,9,10), write a program to print the first half values in one line and the last half values in one line._**

---

### Hints:

> **_Use [n1:n2] notation to get a slice from a tuple._**

---

**Main Author's Solution: Python 2**

```python
tp = (1,2,3,4,5,6,7,8,9,10)
tp1 = tp[:5]
tp2 = tp[5:]
print tp1
print tp2
```

---

**My Solution: Python 3**

```python
tpl = (1,2,3,4,5,6,7,8,9,10)

for i in range(0,5):
    print(tpl[i],end = ' ')
print()
for i in range(5,10):
    print(tpl[i],end = ' ')
```

**OR**

```python
tpl = (1,2,3,4,5,6,7,8,9,10)
lst1,lst2 = [],[]

for i in range(0,5):
    lst1.append(tpl[i])

for i in range(5,10):
    lst2.append(tpl[i])

print(lst1)
print(lst2)
```

**OR**

```python

'''
Solution by: CoffeeBrakeInc
'''

tup = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
lt = int(len(tup)/2)
print(tup[:lt], tup[lt:])
```

---

**OR**

```python

'''
Solution by: AasaiAlangaram
'''

tp = (1,2,3,4,5,6,7,8,9,10)

print('The Original Tuple:',tp)

[print('Splitted List :{List}'.format(List = tp[x:x+5])) for x in range(0,len(tp),5)]

```

---

# Question 39

### **Question:**

> **_Write a program to generate and print another tuple whose values are even numbers in the given tuple (1,2,3,4,5,6,7,8,9,10)._**

---

### Hints:

> **_Use "for" to iterate the tuple. Use tuple() to generate a tuple from a list._**

---

**Main Author's Solution: Python 2**

```python
tp = (1,2,3,4,5,6,7,8,9,10)
li = list()
for i in tp:
	if tp[i]%2 == 0:
		li.append(tp[i])

tp2 = tuple(li)
print tp2
```

---

**My Solution: Python 3**

```python
tpl = (1,2,3,4,5,6,7,8,9,10)
tpl1 = tuple(i for i in tpl if i%2 == 0)
print(tpl1)
```

**OR**

```python
tpl = (1,2,3,4,5,6,7,8,9,10)
tpl1 = tuple(filter(lambda x : x%2==0,tpl))  # Lambda function returns True if found even element.
                                             # Filter removes data for which function returns False
print(tpl1)
```

---

# Question 40

### **Question:**

> **_Write a program which accepts a string as input to print "Yes" if the string is "yes" or "YES" or "Yes", otherwise print "No"._**

---

### Hints:

> **_Use if statement to judge condition._**

---

**Main Author's Solution: Python 2**

```python
s= raw_input()
if s=="yes" or s=="YES" or s=="Yes":
    print "Yes"
else:
    print "No"
```

---

**Solution: Python 3**

```python
'''
Solution by: Seawolf159
'''
text = input("Please type something. --> ")
if text == "yes" or text == "YES" or text == "Yes":
    print("Yes")
else:
    print("No")
```

---

```python
'''
Solution by: AasaiAlangaram
'''
input = input('Enter string:')
output = ''.join(['Yes' if input == 'yes' or input =='YES' or input =='Yes' else 'No' ])
print(str(output))
```
----------------
```
Solution by: Prashanth
'''
x = str(input().lower())
if x == 'yes':
    print('Yes')
else:
    print('No')
```
--------

# Question 41

### **Question:**

> **_Write a program which can map() to make a list whose elements are square of elements in [1,2,3,4,5,6,7,8,9,10]._**

---

### Hints:

> **_Use map() to generate a list.Use lambda to define anonymous functions._**

---

**Main Author's Solution: Python 2**

```python
li = [1,2,3,4,5,6,7,8,9,10]
squaredNumbers = map(lambda x: x**2, li)
print squaredNumbers
```

---

**My Solution: Python 3**

```python
# No different way of code is written as the requirment is specificly mentioned in problem description

li = [1,2,3,4,5,6,7,8,9,10]
squaredNumbers = map(lambda x: x**2, li)  # returns map type object data
print(list(squaredNumbers))               # converting the object into list
```

---

# Question 42

### **Question:**

> **_Write a program which can map() and filter() to make a list whose elements are square of even number in [1,2,3,4,5,6,7,8,9,10]._**

---

### Hints:

> **_Use map() to generate a list.Use filter() to filter elements of a list.Use lambda to define anonymous functions._**

---

**Main Author's Solution: Python 2**

```python
li = [1,2,3,4,5,6,7,8,9,10]
evenNumbers = map(lambda x: x**2, filter(lambda x: x%2==0, li))
print evenNumbers
```

---

**My Solution: Python 3**

```python
def even(x):
    return x%2==0

def squer(x):
    return x*x

li = [1,2,3,4,5,6,7,8,9,10]
li = map(squer,filter(even,li))   # first filters number by even number and the apply map() on the resultant elements
print(list(li))
```

---

# Question 43

### **Question:**

> **_Write a program which can filter() to make a list whose elements are even number between 1 and 20 (both included)._**

---

### Hints:

> **_Use filter() to filter elements of a list.Use lambda to define anonymous functions._**

---

**Main Author's Solution: Python 2**

```python
evenNumbers = filter(lambda x: x%2==0, range(1,21))
print evenNumbers
```

---

**My Solution: Python 3**

```python
def even(x):
    return x%2==0

evenNumbers = filter(even, range(1,21))
print(list(evenNumbers))
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_10.md "Day 10")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_12.md "Day 12")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_12.md
================================================
# Question 44

### **Question:**

> **_Write a program which can map() to make a list whose elements are square of numbers between 1 and 20 (both included)._**

---

### Hints:

> **_Use map() to generate a list. Use lambda to define anonymous functions._**

---

**Main Author's Solution: Python 2**

```python
squaredNumbers = map(lambda x: x**2, range(1,21))
print squaredNumbers
```

---

**My Solution: Python 3**

```python
def sqr(x):
    return x*x

squaredNumbers = list(map(sqr, range(1,21)))
print (squaredNumbers)
```

---

# Question 45

### **Question:**

> **_Define a class named American which has a static method called printNationality._**

---

### Hints:

> **_Use @staticmethod decorator to define class static method.There are also two more methods.To know more, go to this [link](https://realpython.com/blog/python/instance-class-and-static-methods-demystified/)._**

---

**Main Author's Solution: Python 2**

```python
class American(object):
    @staticmethod
    def printNationality():
        print "America"

anAmerican = American()
anAmerican.printNationality()
American.printNationality()
```

---

**My Solution: Python 3**

```python
class American():
    @staticmethod
    def printNationality():
        print("I am American")

american = American()
american.printNationality()   # this will not run if @staticmethod does not decorates the function.
                              # Because the class has no instance.

American.printNationality()   # this will run even though the @staticmethod
                              # does not decorate printNationality()
```

---

# Question 46

### **Question:**

> **_Define a class named American and its subclass NewYorker._**

---

### Hints:

> **Use class Subclass(ParentClass) to define a subclass.\***

---

**Main Author's Solution: Python 2**

```python
class American(object):
    pass

class NewYorker(American):
    pass

anAmerican = American()
aNewYorker = NewYorker()
print anAmerican
print aNewYorker
```

---

**My Solution: Python 3**

```python
class American():
    pass

class NewYorker(American):
    pass

american = American()
newyorker = NewYorker()

print(american)
print(newyorker)
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_11.md "Day 11")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_13.md "Day 13")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_13.md
================================================
# Question 47

### **Question**

> **_Define a class named Circle which can be constructed by a radius. The Circle class has a method which can compute the area._**

---

### Hints

> **_Use def methodName(self) to define a method._**

---

**Main author's Solution: Python 2**

```python
class Circle(object):
    def __init__(self, r):
        self.radius = r

    def area(self):
        return self.radius**2*3.14

aCircle = Circle(2)
print aCircle.area()
```

---

**My Solution: Python 3**

```python
class Circle():
    def __init__(self,r):
        self.radius = r

    def area(self):
        return 3.1416*(self.radius**2)


circle = Circle(5)
print(circle.area())
```

---

# Question 48

### **Question**

> **_Define a class named Rectangle which can be constructed by a length and width. The Rectangle class has a method which can compute the area._**

---

### Hints

> **_Use def methodName(self) to define a method._**

---

**Main author's Solution: Python 2**

```python
class Rectangle(object):
    def __init__(self, l, w):
        self.length = l
        self.width  = w

    def area(self):
        return self.length*self.width

aRectangle = Rectangle(2,10)
print aRectangle.area()

```

---

**My Solution: Python 3**

```python
class Rectangle():
    def __init__(self,l,w):
        self.length = l
        self.width = w

    def area(self):
        return self.length*self.width


rect = Rectangle(2,4)
print(rect.area())

```

---

# Question 49

### **Question**

> **_Define a class named Shape and its subclass Square. The Square class has an init function which takes a length as argument. Both classes have a area function which can print the area of the shape where Shape's area is 0 by default._**

---

### Hints

> **_To override a method in super class, we can define a method with the same name in the super class._**

---

**Main author's Solution: Python 2**

```python
class Shape(object):
    def __init__(self):
        pass

    def area(self):
        return 0

class Square(Shape):
    def __init__(self, l):
        Shape.__init__(self)
        self.length = l

    def area(self):
        return self.length*self.length

aSquare= Square(3)
print aSquare.area()
```

---

**My Solution: Python 3**

```python
class Shape():
    def __init__(self):
        pass

    def area(self):
        return 0

class Square(Shape):
    def __init__(self,length = 0):
        Shape.__init__(self)
        self.length = length

    def area(self):
        return self.length*self.length

Asqr = Square(5)
print(Asqr.area())      # prints 25 as given argument

print(Square().area())  # prints zero as default area
```

---

# Question 50

### **Question**

> **_Please raise a RuntimeError exception._**

---

### Hints

> **_UUse raise() to raise an exception._**

---

**Solution:**

```python
raise RuntimeError('something wrong')
```

---

## Conclusion

**_Well It seems that the above problems are very much focused on basic concpets and implimantation of object oriented programming.As the concepts are not about to solve any functional problem rather design the structure , so both codes are very much similar in there implimantation part._**

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_12.md "Day 12")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_14.md "Day 14")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_14.md
================================================

# Question 51

### **Question**

> ***Write a function to compute 5/0 and use try/except to catch the exceptions.***

----------------------
### Hints 
> ***Use try/except to catch exceptions.***

----------------------

**Main author's Solution: Python 2**
```python
def throws():
    return 5/0

try:
    throws()
except ZeroDivisionError:
    print "division by zero!"
except Exception, err:
    print 'Caught an exception'
finally:
    print 'In finally block for cleanup'
```
----------------
**My Solution: Python 3**
```python
def divide():
    return 5/0

try:
    divide()
except ZeroDivisionError as ze:
    print("Why on earth you are dividing a number by ZERO!!")
except:
    print("Any other exception")

```
---------------------


# Question 52

### **Question**

> ***Define a custom exception class which takes a string message as attribute.***

----------------------
### Hints 
> ***To define a custom exception, we need to define a class inherited from Exception.***

----------------------

**Main author's Solution: Python 2**
```python
class MyError(Exception):
    """My own exception class

    Attributes:
        msg  -- explanation of the error
    """

    def __init__(self, msg):
        self.msg = msg

error = MyError("something wrong")

```
----------------
**My Solution: Python 3**
```python

class CustomException(Exception):
    """Exception raised for custom purpose

    Attributes:
        message -- explanation of the error
    """

    def __init__(self, message):
        self.message = message


num = int(input())

try:
    if num < 10:
        raise CustomException("Input is less than 10")
    elif num > 10:
        raise CustomException("Input is grater than 10")
except CustomException as ce:
    print("The error raised: " + ce.message)

```
---------------------


# Question 53

### **Question**

> ***Assuming that we have some email addresses in the "username@companyname.com" format, please write program to print the user name of a given email address. Both user names and company names are composed of letters only.***

> ***Example:
If the following email address is given as input to the 
program:***
```
john@google.com
```
> ***Then, the output of the program should be:***
```
john
```
> ***In case of input data being supplied to the question, it should be assumed to be a console input.***

----------------------
### Hints 
> ***Use \w to match letters.***

----------------------

**Main author's Solution: Python 2**
```python
import re
emailAddress = raw_input()
pat2 = "(\w+)@((\w+\.)+(com))"
r2 = re.match(pat2,emailAddress)
print r2.group(1)
```
----------------
**My Solution: Python 3**
```python
email = "john@google.com"
email = email.split('@')
print(email[0])
```
---------------------
**OR**
```python
import re

email = "john@google.com elise@python.com"
pattern = "(\w+)@\w+.com"
ans = re.findall(pattern,email)
print(ans)
```


[***go to previous day***](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_13.md "Day 13")

[***go to next day***](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_15.md "Day 15")

[***Discussion***](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)

================================================
FILE: 1.python-basic/Python-100/Status/Day_15.md
================================================
# Question 54

### **Question**

> **_Assuming that we have some email addresses in the "username@companyname.com" format, please write program to print the company name of a given email address. Both user names and company names are composed of letters only._**

> **_Example:
> If the following email address is given as input to the program:_**

```
john@google.com
```

> **_Then, the output of the program should be:_**

```
google
```

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

### Hints

> **_Use \w to match letters._**

---

**Main author's Solution: Python 2**

```python
import re
emailAddress = raw_input()
pat2 = "(\w+)@(\w+)\.(com)"
r2 = re.match(pat2,emailAddress)
print r2.group(2)
```

---

**My Solution: Python 3**

```python
import re

email = "john@google.com elise@python.com"
pattern = "\w+@(\w+).com"
ans = re.findall(pattern,email)
print(ans)
```

---

# Question 55

### **Question**

> **_Write a program which accepts a sequence of words separated by whitespace as input to print the words composed of digits only._**

> **_Example:
> If the following words is given as input to the program:_**

```
2 cats and 3 dogs.
```

> **_Then, the output of the program should be:_**

```
['2', '3']
```

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

### Hints

> **_Use re.findall() to find all substring using regex._**

---

**Main author's Solution: Python 2**

```python
import re
s = raw_input()
print re.findall("\d+",s)
```

---

**My Solution: Python 3**

```python
import re

email = input()
pattern = "\d+"
ans = re.findall(pattern,email)
print(ans)
```

**OR**

```python
email = input().split()
ans = []
for word in email:
    if word.isdigit():     # can also use isnumeric() / isdecimal() function instead
       ans.append(word)
print(ans)
```

**OR**

```python
email = input().split()
ans = [word for word in email if word.isdigit()]  # using list comprehension method
print(ans)
```

---

# Question 56

### **Question**

> **_Print a unicode string "hello world"._**

---

### Hints

> **_Use u'strings' format to define unicode string._**

---

**Main author's Solution: Python 2**

```python
unicodeString = u"hello world!"
print unicodeString
```

---

# Question 57

### **Question**

> **_Write a program to read an ASCII string and to convert it to a unicode string encoded by utf-8._**

---

### Hints

> **_Use unicode()/encode() function to convert._**

---

**Main author's Solution: Python 2**

```python
s = raw_input()
u = unicode( s ,"utf-8")
print u
```

---

**My Solution: Python 3**

```python
s = input()
u = s.encode('utf-8')
print(u)
```

---

# Question 58

### **Question**

> **_Write a special comment to indicate a Python source code file is in unicode._**

---

### Hints

> **_Use unicode() function to convert._**

---

**Solution:**

```python
# -*- coding: utf-8 -*-
```

---

# Question 59

### **Question**

> **_Write a program to compute 1/2+2/3+3/4+...+n/n+1 with a given n input by console (n>0)._**

> **_Example:
> If the following n is given as input to the program:_**

```
5
```

> **_Then, the output of the program should be:_**

```
3.55
```

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

### Hints

> **_Use float() to convert an integer to a float.Even if not converted it wont cause a problem because python by default understands the data type of a value_**

---

**Main author's Solution: Python 2**

```python
n=int(raw_input())
sum=0.0
for i in range(1,n+1):
    sum += float(float(i)/(i+1))
print sum
```

---

**My Solution: Python 3**

```python
n = int(input())
sum = 0
for i in range(1, n+1):
    sum+= i/(i+1)
print(round(sum, 2))  # rounded to 2 decimal point
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_14.md "Day 14")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_16.md "Day 16")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_16.md
================================================
# Question 60

### **Question**

> **_Write a program to compute:_**

```
f(n)=f(n-1)+100 when n>0
and f(0)=0
```

> **_with a given n input by console (n>0)._**

> **_Example:
> If the following n is given as input to the program:_**

```
5
```

> **_Then, the output of the program should be:_**

```
500
```

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

### Hints

> **_We can define recursive function in Python._**

---

**Main author's Solution: Python 2**

```python
def f(n):
    if n==0:
        return 0
    else:
        return f(n-1)+100

n=int(raw_input())
print f(n)
```

---

**My Solution: Python 3**

```python
def f(n):
    if n == 0:
        return 0
    return f(n-1) + 100

n = int(input())
print(f(n))
```

---

# Question 61

### **Question**

> **_The Fibonacci Sequence is computed based on the following formula:_**

```
f(n)=0 if n=0
f(n)=1 if n=1
f(n)=f(n-1)+f(n-2) if n>1
```

> **_Please write a program to compute the value of f(n) with a given n input by console._**

> **_Example:
> If the following n is given as input to the program:_**

```
7
```

> **_Then, the output of the program should be:_**

```
13
```

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

### Hints

> **_We can define recursive function in Python._**

---

**Main author's Solution: Python 2**

```python
def f(n):
    if n == 0: return 0
    elif n == 1: return 1
    else: return f(n-1)+f(n-2)

n=int(raw_input())
print f(n)
```

---

**My Solution: Python 3**

```python
def f(n):
    if n < 2:
        return n
    return f(n-1) + f(n-2)

n = int(input())
print(f(n))
```

---

# Question 62

### **Question**

> **_The Fibonacci Sequence is computed based on the following formula:_**

```
f(n)=0 if n=0
f(n)=1 if n=1
f(n)=f(n-1)+f(n-2) if n>1
```

> **_Please write a program to compute the value of f(n) with a given n input by console._**

> **_Example:
> If the following n is given as input to the program:_**

```
7
```

> **_Then, the output of the program should be:_**

```
0,1,1,2,3,5,8,13
```

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

### Hints

> **_We can define recursive function in Python.
> Use list comprehension to generate a list from an existing list.
> Use string.join() to join a list of strings._**

---

**Main author's Solution: Python 2**

```python
def f(n):
    if n == 0: return 0
    elif n == 1: return 1
    else: return f(n-1)+f(n-2)

n=int(raw_input())
values = [str(f(x)) for x in range(0, n+1)]
print ",".join(values)

```

---

**My Solution: Python 3**

```python
def f(n):
    if n < 2:
        fibo[n] = n
        return fibo[n]
    fibo[n] = f(n-1) + f(n-2)
    return fibo[n]

n = int(input())
fibo = [0]*(n+1)  # initialize a list of size (n+1)
f(n)              # call once and it will set value to fibo[0-n]
fibo = [str(i) for i in fibo]   # converting integer data to string type
ans = ",".join(fibo)    # joining all string element of fibo with ',' character
print(ans)

```
---
```python

'''Solution by: popomaticbubble
'''
def fibo(n):
    if n < 2: return n
	return fibo(n-1)+fibo(n-2)

def print_fiblist(n):
    fib_list = [(str(fibo(i))) for i in range(0, n+1)]
    return print(",".join(fib_list))
n = int(input())
print_fiblist(n)

```

---

# Question 63

### **Question**

> **_Please write a program using generator to print the even numbers between 0 and n in comma separated form while n is input by console._**

> **_Example:
> If the following n is given as input to the program:_**

```
10
```

> **_Then, the output of the program should be:_**

```
0,2,4,6,8,10
```

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

### Hints

> **_Use yield to produce the next value in generator._**

---

**Solution:**

```python
def EvenGenerator(n):
    i=0
    while i<=n:
        if i%2==0:
            yield i
        i+=1


n=int(raw_input())
values = []
for i in EvenGenerator(n):
    values.append(str(i))

print ",".join(values)

```

**OR**

```python
# Solution by: StartZer0
n = int(input())

for i in range(0, n+1, 2):
  if i < n - 1:
    print(i, end = ',' )
  else:
    print(i)
```

---

# Question 64

### **Question**

> **_Please write a program using generator to print the numbers which can be divisible by 5 and 7 between 0 and n in comma separated form while n is input by console._**

> **_Example:
> If the following n is given as input to the program:_**

```
100
```

> **_Then, the output of the program should be:_**

```
0,35,70
```

> **_In case of input data being supplied to the question, it should be assumed to be a console input._**

---

### Hints

> **_Use yield to produce the next value in generator._**

---

**Main author's Solution: Python 2**

```python
def NumGenerator(n):
    for i in range(n+1):
        if i%5==0 and i%7==0:
            yield i

n=int(raw_input())
values = []
for i in NumGenerator(n):
    values.append(str(i))

print ",".join(values)
```

---

**My Solution: Python 3**

```python
def generate(n):
    for i in range(n+1):
        if i % 35 == 0:    # 5*7 = 35, if a number is divisible by a & b then it is also divisible by a*b
            yield i

n = int(input())
resp = [str(i) for i in generate(n)]
print(",".join(resp))

```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_15.md "Day 15")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_17.md "Day 17")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_17.md
================================================
# Question 65

### **Question**

> **_Please write assert statements to verify that every number in the list [2,4,6,8] is even._**

---

### Hints

> **_Use "assert expression" to make assertion._**

---

**Main author's Solution: Python 2**

```python
li = [2,4,6,8]
for i in li:
    assert i%2==0
```

---

**My Solution: Python 3**

```python
data = [2,4,5,6]
for i in data:
    assert i%2 == 0, "{} is not an even number".format(i)
```

---

# Question 66

### **Question**

> **_Please write a program which accepts basic mathematic expression from console and print the evaluation result._**

> **_Example:
> If the following n is given as input to the program:_**

```
35 + 3
```

> **_Then, the output of the program should be:_**

```
38
```

---

### Hints

> **_Use eval() to evaluate an expression._**

---

**Main author's Solution: Python 2**

```python
expression = raw_input()
print eval(expression)
```

---

**My Solution: Python 3**

```python
expression = input()
ans = eval(expression)
print(ans)
```

---

# Question 67

### **Question**

> **_Please write a binary search function which searches an item in a sorted list. The function should return the index of element to be searched in the list._**

---

### Hints

> **_Use if/elif to deal with conditions._**

---

**Main author's Solution: Python 2**

```python
import math
def bin_search(li, element):
    bottom = 0
    top = len(li)-1
    index = -1
    while top>=bottom and index==-1:
        mid = int(math.floor((top+bottom)/2.0))
        if li[mid]==element:
            index = mid
        elif li[mid]>element:
            top = mid-1
        else:
            bottom = mid+1

    return index

li=[2,5,7,9,11,17,222]
print bin_search(li,11)
print bin_search(li,12)

```

---

**My Solution: Python 3**

```python
#to be written

```
**Solution by ulmasovjafarbek: Python 3**
```python
def binary_search(lst, item):
    low = 0
    high = len(lst) - 1
    
    while low <= high:
        mid = round((low + high) / 2)
        
        if lst[mid] == item:
            return mid
        elif lst[mid] > item:
            high = mid - 1
        else:
            low = mid + 1
    return None
    
lst = [1,3,5,7,]
print(binary_search(lst, 9))   
```
---

**Solution by AasaiAlangaram: Python 3**

```python
def binary_search_Ascending(array, target):
    lower = 0
    upper = len(array)
    print('Array Length:',upper)
    while lower < upper:
        x = (lower + upper) // 2
        print('Middle Value:',x)
        value = array[x]
        if target == value:
            return x
        elif target > value:
            lower = x
        elif target < value:
            upper = x

Array = [1,5,8,10,12,13,55,66,73,78,82,85,88,99]
print('The Value Found at Index:',binary_search_Ascending(Array, 82))

```

---

**Solution by yuan1z: Python 3**

```python
idx = 0
def bs(num,num_list):
    global idx
    if (len(num_list) == 1):
        if num_list[0] == num:
            return idx
        else:
            return "No exit in the list"
    elif num in num_list[:len(num_list)//2]:
        return bs(num,num_list[:len(num_list)//2])
    else:
        idx += len(num_list)//2
    return bs(num,num_list[len(num_list)//2:])

print(bs(66,[1,5,8,10,12,13,55,66,73,78,82,85,88,99,100]))

```

---

# Question 68

### **Question**

> **_Please generate a random float where the value is between 10 and 100 using Python module._**

---

### Hints

> **_Use random.random() to generate a random float in [0,1]._**

---

**Main author's Solution: Python 2**

```python
import random
print random.random()*100
```

---

**My Solution: Python 3**

```python
import random
rand_num = random.uniform(10,100)
print(rand_num)
```

---

# Question 69

### **Question**

> **_Please generate a random float where the value is between 5 and 95 using Python module._**

---

### Hints

> **_Use random.random() to generate a random float in [0,1]._**

---

**Main author's Solution: Python 2**

```python
import random
print random.random()*100-5
```

---

**My Solution: Python 3**

```python
import random
rand_num = random.uniform(5,95)
print(rand_num)
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_16.md "Day 16")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_18.md "Day 18")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_18.md
================================================
# Question 70

### **Question**

> **_Please write a program to output a random even number between 0 and 10 inclusive using random module and list comprehension._**

---

### Hints

> **_Use random.choice() to a random element from a list._**

---

**Main author's Solution: Python 2**

```python
li = [2,4,6,8]
import random
print random.choice([i for i in range(11) if i%2==0])
```

---

**My Solution: Python 3**

```python
import random
resp = [i for i in range(0,11,2)]
print(random.choice(resp))
```

---

# Question 71

### **Question**

> **_Please write a program to output a random number, which is divisible by 5 and 7, between 10 and 150 inclusive using random module and list comprehension._**

---

### Hints

> **_Use random.choice() to a random element from a list._**

---

**Main author's Solution: Python 2**

```python
import random
print random.choice([i for i in range(10,151) if i%5==0 and i%7==0])
```

---

**My Solution: Python 3**

```python
import random
resp = [i for i in range(10,151) if i % 35 == 0 ]
print(random.choice(resp))
```

---

# Question 72

### **Question**

> **_Please write a program to generate a list with 5 random numbers between 100 and 200 inclusive._**

---

### Hints

> **_Use random.sample() to generate a list of random values._**

---

**Main author's Solution: Python 2**

```python

import random
print random.sample(range(100,201), 5)
```

---

**My Solution: Python 3**

```python
import random
resp = random.sample(range(100,201),5)
print(resp)
```

---

# Question 73

### **Question**

> **_Please write a program to randomly generate a list with 5 even numbers between 100 and 200 inclusive._**

---

### Hints

> **_Use random.sample() to generate a list of random values._**

---

**Main author's Solution: Python 2**

```python

import random
print random.sample([i for i in range(100,201) if i%2==0], 5)

```

---

**My Solution: Python 3**

```python
import random
resp = random.sample(range(100,201,2),5)
print(resp)
```

---

# Question 74

### **Question**

> **_Please write a program to randomly generate a list with 5 numbers, which are divisible by 5 and 7 , between 1 and 1000 inclusive._**

---

### Hints

> **_Use random.sample() to generate a list of random values._**

---

**Main author's Solution: Python 2**

```python

import random
print random.sample([i for i in range(1,1001) if i%5==0 and i%7==0], 5)
```

---

**My Solution: Python 3**

```python
import random
lst = [i for i in range(1,1001) if i%35 == 0]
resp = random.sample(lst,5)
print(resp)
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_17.md "Day 17")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_19.md "Day 19")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_19.md
================================================
# Question 75

### **Question**

> **_Please write a program to randomly print a integer number between 7 and 15 inclusive._**

---

### Hints

> **_Use random.randrange() to a random integer in a given range._**

---

**Solution:**

```python
import random
print random.randrange(7,16)
```

---

# Question 76

### **Question**

> **_Please write a program to compress and decompress the string "hello world!hello world!hello world!hello world!"._**

---

### Hints

> **_Use zlib.compress() and zlib.decompress() to compress and decompress a string._**

---

**Solution:**

```python
import zlib
s = 'hello world!hello world!hello world!hello world!'
t = zlib.compress(s)
print t
print zlib.decompress(t)
```
---
```python
'''Solution by: anas1434 
'''
s = 'hello world!hello world!hello world!hello world!'
# In Python 3 zlib.compress() accepts only DataType <bytes>
y = bytes(s, 'utf-8')
x = zlib.compress(y)
print(x)
print(zlib.decompress(x))
```
---

# Question 77

### **Question**

> **_Please write a program to print the running time of execution of "1+1" for 100 times._**

---

### Hints

> **_Use timeit() function to measure the running time._**

---

**Main author's Solution: Python 2**

```python

from timeit import Timer
t = Timer("for i in range(100):1+1")
print t.timeit()
```

---

**My Solution: Python 3**

```python
import datetime

before = datetime.datetime.now()
for i in range(100):
    x = 1 + 1
after = datetime.datetime.now()
execution_time = after - before
print(execution_time.microseconds)
```

**OR**

```python
import time

before = time.time()
for i in range(100):
    x = 1 + 1
after = time.time()
execution_time = after - before
print(execution_time)
```

---

# Question 78

### **Question**

> **_Please write a program to shuffle and print the list [3,6,7,8]._**

---

### Hints

> **_Use shuffle() function to shuffle a list._**

---

**Main author's Solution: Python 2**

```python

from random import shuffle
li = [3,6,7,8]
shuffle(li)
print li

```

---

**My Solution: Python 3**

```python
import random

lst = [3,6,7,8]
random.shuffle(lst)
print(lst)
```

**OR**

```python
import random

# shuffle with a chosen seed
lst = [3,6,7,8]
seed = 7
random.Random(seed).shuffle(lst)
print(lst)
```

---

# Question 79

### **Question**

> **_Please write a program to generate all sentences where subject is in ["I", "You"] and verb is in ["Play", "Love"] and the object is in ["Hockey","Football"]._**

---

### Hints

> **_Use list[index] notation to get a element from a list._**

---

**Main author's Solution: Python 2**

```python

subjects=["I", "You"]
verbs=["Play", "Love"]
objects=["Hockey","Football"]
for i in range(len(subjects)):
    for j in range(len(verbs)):
        for k in range(len(objects)):
            sentence = "%s %s %s." % (subjects[i], verbs[j], objects[k])
            print sentence
```

---

**My Solution: Python 3**

```python
subjects=["I", "You"]
verbs=["Play", "Love"]
objects=["Hockey","Football"]

for sub in subjects:
    for verb in verbs:
        for obj in objects:
            print("{} {} {}".format(sub,verb,obj))
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_18.md "Day 18")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_20.md "Day 20")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_20.md
================================================
# Question 80

### **Question**

> **_Please write a program to print the list after removing even numbers in [5,6,77,45,22,12,24]._**

---

### Hints

> **_Use list comprehension to delete a bunch of element from a list._**

---

**Main author's Solution: Python 2**

```python
li = [5,6,77,45,22,12,24]
li = [x for x in li if x%2!=0]
print li
```

---

**My Solution: Python 3**

```python
def isEven(n):
    return n%2!=0

li = [5,6,77,45,22,12,24]
lst = list(filter(isEven,li))
print(lst)
```

**OR**

```python
li = [5,6,77,45,22,12,24]
lst = list(filter(lambda n:n%2!=0,li))
print(lst)
```

---

# Question 81

### **Question**

> **_By using list comprehension, please write a program to print the list after removing numbers which are divisible by 5 and 7 in [12,24,35,70,88,120,155]._**

---

### Hints

> **_Use list comprehension to delete a bunch of element from a list._**

---

**Main author's Solution: Python 2**

```python
li = [12,24,35,70,88,120,155]
li = [x for x in li if x%5!=0 and x%7!=0]
print li
```

---

**My Solution: Python 3**

```python
li = [12,24,35,70,88,120,155]
li = [x for x in li if x % 35!=0]
print(li)
```

---

# Question 82

### **Question**

> **_By using list comprehension, please write a program to print the list after removing the 0th, 2nd, 4th,6th numbers in [12,24,35,70,88,120,155]._**

---

### Hints

> **_Use list comprehension to delete a bunch of element from a list.
> Use enumerate() to get (index, value) tuple._**

---

**Main author's Solution: Python 2**

```python

li = [12,24,35,70,88,120,155]
li = [x for (i,x) in enumerate(li) if i%2!=0]
print li
```

---

**My Solution: Python 3**

```python
li = [12,24,35,70,88,120,155]
li = [li[i] for i in range(len(li)) if i%2 != 0]
print(li)
```

---

# Question 83

### **Question**

> **_By using list comprehension, please write a program to print the list after removing the 2nd - 4th numbers in [12,24,35,70,88,120,155]._**

---

### Hints

> **_Use list comprehension to delete a bunch of element from a list.
> Use enumerate() to get (index, value) tuple._**

---

**Main author's Solution: Python 2**

```python

li = [12,24,35,70,88,120,155]
li = [x for (i,x) in enumerate(li) if i<3 or 4<i]
print li

```

---

**My Solution: Python 3**

```python
#to be written
li = [12,24,35,70,88,120,155]
li = [li[i] for i in range(len(li)) if i<3 or 4<i]
print(li)
```

---

# Question 84

### **Question**

> **_By using list comprehension, please write a program generate a 3\*5\*8 3D array whose each element is 0._**

---

### Hints

> **_Use list comprehension to make an array._**

---

**Solution:**

```python
array = [[ [0 for col in range(8)] for col in range(5)] for row in range(3)]
print array
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_19.md "Day 19")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_21.md "Day 21")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_21.md
================================================
# Question 85

### **Question**

> **_By using list comprehension, please write a program to print the list after removing the 0th,4th,5th numbers in [12,24,35,70,88,120,155]._**

---

### Hints

> **_Use list comprehension to delete a bunch of element from a list.Use enumerate() to get (index, value) tuple._**

---

**Main author's Solution: Python 2**

```python
li = [12,24,35,70,88,120,155]
li = [x for (i,x) in enumerate(li) if i not in (0,4,5)]
print li
```
---

**My Solution: Python 3**

```python
li = [12,24,35,70,88,120,155]
li = [li[i] for i in range(len(li)) if i not in (0,4,5)]
print(li)
```
---
```python
'''Solution by: pratikb0501
'''
li = [12, 24, 35, 70, 88, 120, 155]
print(list(j for i, j in enumerate(li) if i != 0 and i != 4 and i != 5))

```

---

# Question 86

### **Question**

> **_By using list comprehension, please write a program to print the list after removing the value 24 in [12,24,35,24,88,120,155]._**

---

### Hints

> **_Use list's remove method to delete a value._**

---

**Main author's Solution: Python 2**

```python
li = [12,24,35,24,88,120,155]
li = [x for x in li if x!=24]
print li
```

---

**My Solution: Python 3**

```python
li = [12,24,35,24,88,120,155]
li.remove(24)  # this will remove only the first occurrence of 24
print(li)
```

---

# Question 87

### **Question**

> **_With two given lists [1,3,6,78,35,55] and [12,24,35,24,88,120,155], write a program to make a list whose elements are intersection of the above given lists._**

---

### Hints

> **_Use set() and "&=" to do set intersection operation._**

---

**Main author's Solution: Python 2**

```python

set1=set([1,3,6,78,35,55])
set2=set([12,24,35,24,88,120,155])
set1 &= set2
li=list(set1)
print li
```

---

**My Solution: Python 3**

```python
list1 = [1,3,6,78,35,55]
list2 = [12,24,35,24,88,120,155]
set1= set(list1)
set2= set(list2)
intersection = set1 & set2
print(intersection)
```

**OR**

```python
list1 = [1,3,6,78,35,55]
list2 = [12,24,35,24,88,120,155]
set1= set(list1)
set2= set(list2)
intersection = set.intersection(set1,set2)
print(intersection)
```

---

# Question 88

### **Question**

> **_With a given list [12,24,35,24,88,120,155,88,120,155], write a program to print this list after removing all duplicate values with original order reserved._**

---

### Hints

> **_Use set() to store a number of values without duplicate._**

---

**Main author's Solution: Python 2**

```python
def removeDuplicate( li ):
    newli=[]
    seen = set()
    for item in li:
        if item not in seen:
            seen.add( item )
            newli.append(item)

    return newli

li=[12,24,35,24,88,120,155,88,120,155]
print removeDuplicate(li)

```

---

**My Solution: Python 3**

```python
li = [12,24,35,24,88,120,155,88,120,155]
for i in li:
    if li.count(i) > 1:
        li.remove(i)
print(li)
```

**OR**

```python
def removeDuplicate( li ):
    seen = {}  # dictionary
    for item in li:
        if item not in seen:
            seen[item] = True
            yield item

li = [12, 24, 35, 24, 88, 120, 155, 88, 120, 155]
ans = list(removeDuplicate(li))
print(ans)
```

---

# Question 89

### **Question**

> **_Define a class Person and its two child classes: Male and Female. All classes have a method "getGender" which can print "Male" for Male class and "Female" for Female class._**

---

### Hints

> **_Use Subclass(Parentclass) to define a child class._**

---

**Solution:**

```python
class Person(object):
    def getGender( self ):
        return "Unknown"

class Male( Person ):
    def getGender( self ):
        return "Male"

class Female( Person ):
    def getGender( self ):
        return "Female"

aMale = Male()
aFemale= Female()
print aMale.getGender()
print aFemale.getGender()
```

---

[**_go to previous day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_20.md "Day 20")

[**_go to next day_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/blob/master/Status/Day_22.md "Day 22")

[**_Discussion_**](https://github.com/darkprinx/100-plus-Python-programming-exercises-extended/issues/3)


================================================
FILE: 1.python-basic/Python-100/Status/Day_22.md
================================================
# Question 90

### **Question**

> **_Please write a program which count and print the numbers of each character in a string input by console._**

> **_Example:
> If the following string is given as input to the program:_**

```
abcdefgabc
```

> **_Then, the output of the program should be:_**

```
a,2
c,2
b,2
e,1
d,1
g,1
f,1
```

### Hints

> **_Use dict to store key/value pairs.
> Use dict.get() method to lookup a key with default value._**

---

**Main author's Solution: Python 2**

```python
dic = {}
s=raw_input()
for s in s:
    dic[s] = dic.get(s,0)+1
print '\n'.join(['%s,%s' % (k, v) for k, v in dic.items()])
```

---

**My Solution: Python 3**

```python
import string

s = input()
for letter in string.ascii_lowercase:
    cnt = s.count(letter)
    if cnt > 0:
        print("{},{}".format(letter,cnt))
```

**OR**

```python
s = input()
for letter in range(ord('a'),ord('z')+1):    # ord() gets the ascii value of a char
    letter = chr(letter)                     # chr() gets the char of an ascii value
    cnt = s.count(letter)
    if cnt > 0:
        print("{},{}".format(letter,cnt))
```

---

# Question 91

### **Question**

> **_Please write a program which accepts a string from console and print it in reverse order._**

> **Example:
> If the following string is given as input to the program:\***

```
rise to vote sir
```

> **_Then, the output of the program should be:_**

```
ris etov ot esir
```

### Hints

> **_Use list[::-1] to iterate a list in a r