Summary Notes for Anomaly Detection

 Finding unusual events:



Gaussian (Normal) Distribution:



Anomaly detection algorithm:







Developing and evaluating an anomaly detection system:




Anomaly detection v.s. Supervised learning






Anomaly detection is designed to identify novel positive instances that may not resemble any previously observed examples. In contrast, supervised learning uses known positive examples to determine whether future instances are similar to what has been seen before.

When building an anomaly detection algorithm, choosing what features to use ?

Non-gaussian features






The typical development process involves training the model, then reviewing the anomalies in the cross-validation set that the algorithm fails to detect. By examining these missed examples, you can identify potential new features. For instance, if an anomaly exhibits unusually high or low values on a newly engineered feature, that feature can help the algorithm successfully flag similar anomalies.

Comments

Post a Comment

Popular posts from this blog

Analysis of Repeated Measures Data using SAS

Image
 1. Basic Concepts of Repeated Measures: In this basic setup of a completely randomized design with repeated measures, there are two factors, treatments and time. Treatment is called the between-subjects factor because levels of treatment can change only between subjects; all measurements on the same subject will represent the same treatment. Time is called a within-subjects factor because different measurements on the same subject are taken at different times.  In repeated measures experiments, interest centers on (1) how treatment means differ, (2) how treatment means change over time, and (3) how differences between treatment means change over time. 2. Four-step procedure for mixed model analysis: Step 1: Model the mean structure, usually by specification of the fixed effects. Step 2: Specify the covariance structure, between subjects as well as within subjects. Step 3: Fit the mean model accounting for the covariance structure. Step 4: Make statistical inference bas...
Read more

Four essential statistical functions for simulation in SAS

  PDF function : This function is the probability density function. It returns the probability density at a given point for a variety of distributions. (For discrete distribution, the PDF function evaluates the probability mass function.) CDF function : This function is the cumulative distribution function. The CDF returns the probability that an observation from the specified distribution is less than or equal to a particular value. For continuous distributions, this is the area under the PDF up to a certain point. QUANTILE function : This function is closely related to the CDF function, but solves an inverse problem. Given a probability, P, it returns the smallest value,  q , for which CDF( q ) is greater than or equal to P. RAND function : This function generates a random sample from a distribution. In SAS/IML software, use the  RANDGEN subroutine , which fills up an entire matrix at once.
Read more

Medical information for Melanoma, Merkel cell carcinoma and tumor mutation burden

  Background: Melanoma Also called: malignant melanoma The most serious type of skin cancer. Melanoma occurs when the pigment-producing cells that give color to the skin become cancerous. Symptoms might include a new, unusual growth or a change in an existing mole. Melanomas can occur anywhere on the body. Treatment may involve surgery, radiation, medications, or in some cases chemotherapy. Merkel cell carcinoma  (MCC) it is a very rare ,  highly aggressive, neuroendocrine carcinoma of the skin ;  it has a worse survival probability compared with more common cutaneous malignancies, including metastatic melanoma, and has been termed the “most lethal” skin cancer  as   p atients with metastatic MCC have a historical 5-year overall survival (OS) rate of 18% What Is Tumor Mutational Burden? TMB is defined as the frequency of certain mutations within a tumor’s genes.  To be counted towards TMB, mutations have to alter the protein that is made from a gene. ...
Read more