Exploratory data analysis - Revision history

Rice at 19:32, 17 May 2024

2024-05-17T19:32:12Z

← Older revision		Revision as of 19:32, 17 May 2024
Line 35:		Line 35:
	* Min-max scaling: <math>X_{norm} = \frac{X - X_{min}}{X_{max} - X_{min}}</math>		* Min-max scaling: <math>X_{norm} = \frac{X - X_{min}}{X_{max} - X_{min}}</math>
	* Z-score scaling: <math>X_{\text{norm}} = \frac{X - \mu} {\sigma}</math>		* Z-score scaling: <math>X_{\text{norm}} = \frac{X - \mu} {\sigma}</math>

			[[Category:Machine Learning]]

Rice at 06:30, 26 April 2024

2024-04-26T06:30:57Z

← Older revision		Revision as of 06:30, 26 April 2024
Line 1:		Line 1:
	'''Exploratory data analysis (EDA)''' is the first step in the Machine Learning pipeline. It allows us to make informed decisions about tools used to analyze the data.		'''Exploratory data analysis (EDA)''' is the first step in the Machine Learning pipeline. It allows us to make informed decisions about tools used to analyze the data.

	* Look at features of data		= Dataset description =
	* Look at correlated features		During the EDA phase, we can make choices regarding which model to use depending on the [[Dataset#Usage attributes\|attributes of a dataset]].
	* Find trends and unusual characteristics
	Dataset ~~description~~

			= Other tasks =
	EDA also detects '''unwanted values/noise''' that lead to inaccurate predictions.		EDA also detects '''unwanted values/noise''' that lead to inaccurate predictions.

	~~During the EDA phase, we can make choices regarding which model to use depending on the dataset, such as visualizing the data and then checking linearity.e~~

	= Outlier =		= Outlier =

Rice: Rice moved page Exploratory Data Analysis to Exploratory data analysis

2024-04-26T06:12:37Z

Rice moved page Exploratory Data Analysis to Exploratory data analysis

← Older revision		Revision as of 06:12, 26 April 2024
(No difference)

Rice: /* Outlier */

2024-04-05T18:11:50Z

Outlier

← Older revision		Revision as of 18:11, 5 April 2024
Line 26:		Line 26:

	''Skewed data'' tend to have outliers on the side it's skewed to.		''Skewed data'' tend to have outliers on the side it's skewed to.

			= Data Normalization =
			'''Normalization''' is a technique used to scale numeric data to the same scale. It

			* helps in faster convergence
			* prevents some features from dominating others due to scale
			* improves the overall performance of the model

			There are several strategies to do this

			* Min-max scaling: <math>X_{norm} = \frac{X - X_{min}}{X_{max} - X_{min}}</math>
			* Z-score scaling: <math>X_{\text{norm}} = \frac{X - \mu} {\sigma}</math>

Rice at 18:41, 3 April 2024

2024-04-03T18:41:29Z

← Older revision		Revision as of 18:41, 3 April 2024
Line 8:		Line 8:
	EDA also detects '''unwanted values/noise''' that lead to inaccurate predictions.		EDA also detects '''unwanted values/noise''' that lead to inaccurate predictions.

	EDA ~~finds~~ the ~~correlation~~ of ~~attributes of datasets~~, such as ~~linearity~~.		During the EDA phase, we can make choices regarding which model to use depending on the dataset, such as visualizing the data and then checking linearity.e

			= Outlier =
			Outliers in a dataset are those samples that are abnormally extreme (i.e. far from the other data points). They can affect the overall accuracy of the model trained on the data. They are '''anomalies''', causing biased or incorrect data.

			During the EDA phase, these outliers are detected

			* ''Background knowledge'' such as impossible values like negative age
			* ''Visualization'' such as scatter plot
			* [[Summary Statistics#Outliers\|''Data Analysis'']] such as box plot
			* ''ML algorithms'' such as One-Class-SVM

			and strategies are used to deal with them

			* Correct values
			* Delete values

			''Skewed data'' tend to have outliers on the side it's skewed to.

Rice at 18:16, 3 April 2024

2024-04-03T18:16:45Z

← Older revision		Revision as of 18:16, 3 April 2024
Line 4:		Line 4:
	* Look at correlated features		* Look at correlated features
	* Find trends and unusual characteristics		* Find trends and unusual characteristics
			Dataset description

			EDA also detects '''unwanted values/noise''' that lead to inaccurate predictions.

			EDA finds the correlation of attributes of datasets, such as linearity.

Rice: Created page with "'''Exploratory data analysis (EDA)''' is the first step in the Machine Learning pipeline. It allows us to make informed decisions about tools used to analyze the data. * Look at features of data * Look at correlated features * Find trends and unusual characteristics"

2024-04-03T18:13:25Z

Created page with "'''Exploratory data analysis (EDA)''' is the first step in the Machine Learning pipeline. It allows us to make informed decisions about tools used to analyze the data. * Look at features of data * Look at correlated features * Find trends and unusual characteristics"

New page

'''Exploratory data analysis (EDA)''' is the first step in the Machine Learning pipeline. It allows us to make informed decisions about tools used to analyze the data.

* Look at features of data
* Look at correlated features
* Find trends and unusual characteristics