CONVOLUTION NEURAL NETWORK IN DEEP LEARNING/PYTHON/ARTIFICIAL INTELLIGENCE

Convolution Neural Network

• Advantages of Convolutional Neural Network
• Disadvantages of Convolutional Neural Network

The Convolutional Neural Network (CNN) operates on a feed-forward basis, its neuron connectivity pattern inspired by the visual cortex. This CNN deep learning architecture is prominently used in computer vision, an artificial intelligence field focused on interpreting visual data. In this blog the convolutional neural network explanation is done in detail.

A deep learning Convolutional Neural Network is a more advanced version of a conventional artificial neural network that is designed specifically to extract features from datasets that resemble grids. These networks, also referred to as covnets, share parameters between layers. The convolutional layer of a CNN extracts relevant characteristics from input images by applying filters to them. To lessen the computing load, a pooling layer also down samples the image; in the end, the fully connected layer in cnn mainly provides the final predictions. Gradient descent and backpropagation are used to determine how the CNN model determines which filters work best.

It also generally has three layers like Artificial Neural Networks with the same name which are the input layer, hidden layer, and output layer.

Image source original

Input layer: As the name implies, this layer receives input from the real world. Its neuron count aligns with the total number of features within the data (in image scenarios, this equates to the number of pixels).

Hidden layer: After the data passes through the input layer, it moves on to the hidden layers of the neural network. The presence and number of these layers depend on the model and the size of the data set. The number of neurons in each hidden layer can vary, often exceeding the number of inputs. The learning weights and biases are used to calculate the output of each layer by multiplying that output by a matrix derived from the output of the previous layer. An activation function is then applied to that result, which introduces nonlinearity in the network.

Output layer: After processing through the hidden layers, the data is fed into a logistic function such as sigmoid or softmax. These functions convert the results of each class into probability scores and provide a probability estimate for each class. Then data is produced as the output from the output layer. The output layer is the last layer of the neural network that is used to produce the output.

The outcome generated from this process is termed "feedforward." Subsequently, an error calculation takes place using an error function, the error function can be a cross-entropy or square loss error. These functions measure the network's performance or the magnitude of error. Following this, derivatives are computed in a step called backpropagation, primarily aimed at minimizing the loss. 

Convolutional Neural Network Example

Take a look at one such case. Accurately analyzing medical photographs in the face of an increasing number of patient scans presented a significant issue for the healthcare sector. Radiology specialists were overburdened with the volume of cases, which made diagnostic delays more frequent and perhaps dangerous for patients.

Convolutional Neural Networks (CNNs) are a ground-breaking method employed by the healthcare sector to tackle this issue. These advanced artificial intelligence algorithms learned to recognize patterns suggestive of different medical disorders from large datasets of medical pictures, including X-rays, CT scans, and MRI images.

As a new patient underwent imaging procedures, their scans were fed into the CNN machine learning model, which thoroughly analyzed each image pixel by pixel. The CNN’s deep layers of convolutional and pooling operations enabled it to get intricate features and nuances from the images, far surpassing the capabilities of traditional image processing techniques.

By using CNN radiologists from the healthcare industry could expedite the diagnostic process while maintaining a high level of accuracy. The CNN can flag abnormalities like tumors, fractures, and anomalies in organs, providing the healthcare industry and radiologist invaluable insights to guide their further process.

Convolutional Neural Network Working

A convolutional neural network (CNN) is an advanced version of an artificial neural network specifically designed to extract features from grid-like matrix datasets. It finds wide application in image or video problems where data patterns are crucial.

CNN Architecture

The CNN has multiple layers like the input layer, convolutional layer, polling layer, and fully connected layers.

Image source original

The first layer that interacts with the image is the convolutional layer, which is responsible for applying filters to the input image to extract its features. After that, the aggregation layer samples the image to reduce the computational load. Finally, the fully combined layer provides the final forecast. The network uses backpropagation and gradient descent to obtain the most efficient filter.

How Convolutional Layers Works

Convolutional Neural Networks (CNNs) share parameters across layers. Consider an image, which can be visualized as a cuboid with length and width representing the image's dimensions and height depicting the channels (e.g., red, green, and blue channels in typical images). 

Image source original

Consider a scenario where we extract a small segment from an image and apply a neural network, referred to as a filter or kernel, to this segment, generating, for instance, K outputs arranged vertically. Extending this process across the entire image involves sliding this neural network across the image. Consequently, we obtain a modified image, differing in width, height, and depth. Unlike the traditional red, green, and blue channels (RGB), this image now possesses additional channels but reduced width and height. This process is known as Convolution. When the patch size aligns with the image size, it essentially operates as a conventional neural network. This use of smaller patches significantly reduces the number of weights involved.

Note: - the LeNet architecture is a pioneering convolutional neural network developed by Yann LeCun and his colleagues, that laid the foundation for modern deep learning models, that revolutionize various fields including computer vision and pattern recognition.

Let’s look at the mathematics that is involved

Image source original

In this whole convolution process.

• Convolutional layers consist of trainable filters, often called kernels. These filters are characterized by small dimensions in both width and height and correspond to the depth of the input volume, which is typically 3 for image inputs.
• Consider an image of dimensions 34x34x3 for convolution. The size of the filter can be aXaX3, where "a" can vary as 3, 5, or 7, but must be smaller than the size of the image.
• In a forward filter, each filter moves through the entire input volume step by step. This movement is called a step, which for large images can take values like 2, 3, or 4. At each step, the network calculates the dot product between the kernel weights and the patch extracted from the input volume.
• As the filters move over the input, a 2-D output is created for each filter. These outputs are then stacked together to form an output volume with a depth equal to the number of filters used. This process allows the network to learn and adjust its filters accordingly.

Layers used to build Convolutional Neural Network

A convolutional neural network (CNN) comprises different layers, each transforming one volume into another using a differentiable function. Here's an overview using an example image of dimensions 32 x 32 x 3.

Input Layer: The input layer receives input from the external world, commonly images or image sequences in CNNs. It holds the original image data, the image data have a width of 32, height of 32, and depth of 3.

Convolutional Layers: Here, the input dataset undergoes feature extraction. Kernels or learnable filters, typically 2x2, 3x3, or 5x5 matrices, are applied to the input images. These filters slide over the input data, performing dot products with corresponding patches. The resulting output is termed feature maps. For instance, if there are 12 filters in this layer, the output volume becomes 32x32x12.

Activation Layer: By introducing nonlinearity in the network, this layer applies an element-wise activation function to the output of the convolution layer. Popular activation features include e.g. ReLU (max(0, x)), Tanh, and Leaky ReLU. Despite the activation function, this layer retains its original dimensions of 32 x 32 x 12.

Pooling Layer: Periodically integrated into the CNN, this layer aims to reduce data volume size for faster computation, less memory usage, and overfitting prevention. Common types are max pooling and average pooling. For instance, using 2x2 filters and a stride of 2 in max pooling would yield an output volume of 16x16x12.

Image source original

Flattening: After the convolution and pooling of layers, the resulting feature maps are smoothed into a one-dimensional vector. This reformulation allows them to be sent to a fully connected layer suitable for solving categorical or regression problems.

Fully Connected Layers: This layer, located immediately before the output layer, receives input from the previous layer and performs computations for the final classification or regression task.

Output layer: The output of the fully connected layers is fed into a logistic function that effectively transforms the output of each class into an equivalent probability score indicating its classification probability, such as with sigmoid or softmax functions.

Advantages of Convolutional Neural Network

• Feature Learning: CNNs automatically learn hierarchical representations of features from input data. Convolutional layers work to extract low-level features like edges and textures, on the other hand, deeper layers learn high-level representations like shapes and patterns. This hierarchical feature extraction reduces the need for manual feature engineering.
• Spatial Hierarchies: CNNs preserve the spatial relationships between pixels in images due to their convolutional and pooling operations. This spatial awareness enables them to capture local patterns regardless of their location in the image.
• Parameter Sharing: By using shared weights via convolutional kernels, CNNs able to significantly reduce the number of trainable parameters, making them more efficient in terms of memory and computation. This sharing also allows them to generalize better to new, unseen data.
• Translation invariance: In CNN we have a translational invariance, it can detect patterns from images regardless of their location. It helps CNN in image translation, rotation and distortions.
• Pooling Layers: Through pooling layers (e.g., max pooling), CNNs mainly down-sample the feature maps, which reduces computational requirements while retaining essential information. This down-sampling aids in learning more robust and invariant features.
• Versatility and transfer learning: already trained CNN models on the big dataset can used in other tasks for feature extraction, therefore we use transfer learning in CNN for various aspects.
• State-of-the-Art Performance: image classification using CNN is one of the most used aspects of CNN architecture that it has achieved. Object detection, semantic segmentation, and other tasks also take benefits from CNN and shows their effectiveness in handling complex visual recognition challenges. 
• Scalability: CNNs can handle large-scale and high-resolution images which make them very useful in real-world applications in which we need to analyze large or high-dimensional data.

Disadvantages of Convolutional Neural Network

• Computational Complexity: CNN needs more computational power compared to the traditional approach the computation power increases as CNN becomes deeper, making CNN more computationally expensive and also resource-intensive. This complexity can hinder their deployment in resource-constrained environments.
• Large Data Requirements: CNNs generally require large labeled datasets for effective training. Inadequate data might lead to overfitting or less optimal performance of the model.
• updating & Hyperparameters: Choosing and tweaking suitable hyperparameters (such as kernel size and number of layers) is a challenging and time-consuming process in creating the best CNN architecture.
• Limitated interpretability: It is challenging to understand how and why CNNs create particular predictions because of their complex architecture. This interpretability issue might be problematic in an area where responsibility and openness are required.
• Overfitting: Using smaller datasets in particular, deeper CNN architectures with more parameters are more likely to overfit. Many times, regularization methods are required to avoid overfitting.
• Technical Specifications: Instruction Complex CNN models sometimes require specialized hardware (such as GPUs or TPUs), which not all users or applications may have easy access to or capacity for.
• Preprocessing and Augmenting of Data: To guarantee consistency, cleanliness, and sufficiency, data preparation for CNNs frequently requires substantial preprocessing and augmentation. Complex and time-consuming might be this procedure.
• Lack of Rotation and Scale Invariance: while CNNs possess translation invariance, they may not inherently generalize well to rotations or scale variations in images without additional techniques or augmentation.

Summary

At the forefront of modern computer vision, Convolutional Neural Networks (CNN) provide a sophisticated architecture designed to extract complex patterns from visual data. Basic components such as convolution and pooling layers play a key role in hierarchical feature extraction. Convolutional layers apply filters to input images and detect various visual features such as edges and textures. These features are then compressed by joining the layers, preserving important information while reducing computational complexity. These networks use hierarchical learning and progress through layers to identify increasingly complex patterns by combining lower-level features. The inclusion of non-linear activation functions such as Rectified Linear Units (ReLU) adds flexibility and complexity to the learning of the network, allowing it to model complex relationships within images.

The prowess of CNNs extends across a spectrum of image-related tasks, from CNN model for image classification and object detection to semantic segmentation. Despite their remarkable performance, CNNs demand substantial computational resources and extensive labeled datasets for training. Moreover, interpreting, CNNs stand as a pillar in reshaping machine understanding of visual information, propelling advancements in artificial intelligence, and bolstering applications in diverse domains, from healthcare and autonomous vehicles to robotics and beyond. Graph convolutional network have emerged as a powerful extension of convolutional neural networks, that offer advanced capabilities for processing graph-structured data.

Python Code

ARTIFICIAL NEURAL NETWORK IN DEEP LEARNING/PYTHON/ARTIFICIAL INTELLIGENCE

 Artificial Neural Network

• Architecture Component
• Different Types of Artificial Neural Networks
• Applications of Artificial Neural Networks
• Advantages of Artificial Neural Networks
• Disadvantages of Artificial Neural Networks

Artificial Neural Networks (ANNs) are made to replicate how the human brain works, aiming to simulate its functions. They constitute a subfield of artificial intelligence inspired by biological neural networks. Similar to how neurons are connected in the human brain, artificial neuron networks consist of units connected across multiple layers.

Artificial neurons, often referred to as units, are arranged in layers to form the network structure of an ANN. Depending on how complicated the network is, a layer may have a few hundred or millions of units in it. An artificial neural network AI typically consists of intermediate hidden layers, an output layer, and an input layer. External data is fed into the input layer so that the neural network in artificial intelligence may analyze or learn from it. After that, this data is transformed by processing it through one or many other hidden layers. The output layer receives the changed data at this point and provides the network's reaction to the input data.

Interconnections between most artificial neural networks in machine learning units incorporate weights, dictating the impact of one unit on another. As information moves between units, the artificial neural network algorithm accumulates knowledge about the data, leading to an output from the output layer.

Image source original

What is Artificial Neural Network?

The term “Artificial Neural Network” is derived from the biological neural network that is present in the human brain. Neuron AI in artificial neuron networks has another more common name rather than units which is node most commonly we refer to the node for the neurons.

Belo is the typical image of a human neuron.

In artificial neural networks (ANNs), the structure mirrors the structure of biological neural networks (NNs). In ANNs, inputs correspond to dendrites, nodes represent cell nuclei, weights are analogous to synapses, and outputs resemble axons.

This table illustrates the parallelism between biological and artificial neural networks.

Biological Neural Network	Artificial Neural Network
Dendrites	Inputs
Cell nucleus	Nodes
Synapse	Weights
Axon	Output

The artificial neural network and AI emulates interconnected brain cells and is engineered by programmers. The human brain has approximately 100 billion neurons, and in this, each neuron has connections, ranging from 1,000 to 100,000 associations. An effective way to comprehend an artificial neural network example is by considering a digital gate, which accepts an input and produces an output. 

Real-World Example for Artificial Neural Networks

Let’s look at an example, in a big city there is a financial district, that is grappling with a surge in fraudulent transactions. In this financial district, banks were inundated with cases of identity theft and unauthorized access to accounts, causing financial losses and eroding customer trust.

To tackle this situation data scientist Maya turned to artificial neural networks (ANNs). She gathered vast amounts of transactional data, including user behavior, transaction history, and account details.

Using this data, she trained an artificial neural network machine learning model that can detect patterns indicative of fraudulent activity. The network analyzed each transaction in real-time, flagging suspicious behavior such as unusually large transactions, irregular spending patterns, or multiple failed login attempts.

As the ANN model processed more data, its accuracy increased, enabling it to distinguish between legitimate transactions and fraudulent ones with greater precision. This new development ability allowed banks to identify and prevent fraudulent activity before it could inflict financial harm.

How do Artificial Neural Networks learn?

The artificial neural network Python undergoes training using a training dataset. Consider teaching an ANN to recognize a dog: it's exposed to numerous dog images to learn the identification process. Once the training concludes, the network's ability to correctly identify dog images is tested. By presenting new images, the ANN determines whether they depict dogs or not. Human-provided descriptions verify the network's accuracy. If discrepancies arise, backpropagation comes into play. This method involves specifying the weights of the links in the units of the artificial neural network (ANN) and adjusting them according to the error rate. This iterative process continues until the network can effectively identify dogs from the images, minimizing errors.

The architecture of an artificial neural network

To grasp the workings of a neural network, understanding its components is crucial. A neural network comprises numerous artificial neurons, referred to as units, arranged in layers. Let's explore the different types of layers present in every artificial neural network. The diagram below illustrates the various layers within the network.

Image source original

Input Layer: This layer, as implied, receives inputs in various formats provided by users or programmers.

Hidden Layer: Positioned middle of the input and output layers, the hidden layer undertakes calculations crucial for identifying hidden features and patterns within the dataset or training data.

Output Layer: As the name suggests, this layer acts as an endpoint to present the final output to the users. After a series of transformations enabled by the hidden layer, the input is processed, resulting in the result passed through this layer.

The artificial neural network processes inputs by computing the total weighted sum of inputs along with a bias, represented through a transfer function.

This component calculates the weighted sum, which is then passed to the activation function to generate the output. The activation function specifies whether the node should be activated or not. Only activated nodes persist to the output layer. We have many activation functions, therefore we need to carefully choose the most suitable function.

Different types of Artificial Neural Networks

Feedforward Neural Network: It is one of the most common neural networks it operates in a single direction, in this the data moves from the input to the middle layers and from the middle layers to the output layer it does not have backpropagation.

Convolutional NeuralNetwork (CNN): Similar to the feedforward neural network, the CNN incorporates weighted connections between units or nodes, determining the influence of one unit on another. It employs one or multiple convolutional layers that execute convolutional operations on input data and pass the obtained results to subsequent layers. CNNs are extensively utilized in tasks like image processing and speech recognition, particularly in computer vision applications.

Modular Neural Network: Comprising multiple independent neural networks, a modular neural network operates distinctively, with no interaction among its components. Each network handles a specific sub-task with unique inputs. The advantage lies in its modular approach, breaking down the complex computational processes into little components, and reducing complexity while obtaining the desired output.

Radial Basis Function Neural Network: This network takes advantage of the point-to-center distance and uses only two layers. The first layer maps the radial basis functions in the hidden layer, while the output layer computes the resulting output. These networks are typically used in models that represent underlying patterns or features in data sets.

Recurrent Neural Networks(RNNs): RNNs differ by retaining the output of a layer and feeding it back to the input to enhance outcome prediction. It begins similarly to a feedforward neural network, with each node or unit in subsequent layers remembering information from previous steps, functioning akin to a memory cell to improve computational performance.

We will look at these neural networks in much detail in further chapters.

Applications of Artificial Neural Networks

• Social Media: Artificial neural networks play an important role in many social media platforms, on Facebook you might see suggestions for "people you may know", Facebook does this by analyzing user profiles, interests, existing connections, and more to propose potential acquaintances. Facial recognition is another key application, leveraging convolutional neural networks by identifying facial reference points to match them with database records.
• Marketing and Sales: E-commerce platforms like Amazon or Flipkart employ machine learning to recommend a product to the user based on their browsing history. This personalized marketing spans various sectors like food services, hospitality, movies, and books. Artificial neural networks discern customer preferences, shopping history, and dislikes to tailor marketing strategies accordingly.
• Healthcare: Artificial neural networks find extensive use in healthcare, particularly in cancer detection. In oncology, they train algorithms to identify cancerous tissues at microscopic levels with accuracy akin to trained physicians. Additionally, we can use facial analysis using photos to aid in identifying rare diseases in their early stages, which increases the doctor's diagnostic capabilities, and helps the medical sector globally.
• Personal Assistants: Nowadays we often use digital personal assistants like Siri and Alexa they heavily rely on speech recognition and Natural Language Processing (NLP) which in its core uses artificial neural networks. NLP manages language syntax, semantics, speech accuracy, and ongoing conversations, enabling assistants to interact with users effectively.

Advantages of Artificial Neural Networks

• Flexibility to Complex Patterns: Complex patterns that would be challenging for traditional algorithms to detect and understand are among the patterns in data that ANNs specialize in recognizing.
• Learning Capabilities: They can pick up knowledge and get better with experience. ANNs improve their decision-making and forecast accuracy by iterations and changes to its internal parameters (weights).
• Parallel processing is the ability of Artificial Neural Networks (ANN) to carry out operations concurrently across many neurons or nodes. They may handle large volumes of data and carry out intricate computations more quickly because to this function.
• Extrapolating learnt patterns, artificial neural networks (ANNs) may categorize or forecast based on unknown or unseen input. Overfitting is lessened and model performance for new cases is enhanced by this generalizability.
• Feature Extraction: Artificial Neural Networks (ANNs) may independently extract significant features from raw data in some models, like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), therefore minimizing the requirement for human feature design.
• Fault Tolerance: Because ANNs are dispersed and have redundancy in information representation, they may frequently put up with mistakes or missing data without much sacrificing their overall performance.
• Applications of their versatility include time series analysis, natural language processing, picture and audio recognition, and recommendation systems. Their adaptability highlights their efficacy for a range of issues.
• Performance and accuracy of ANNs can be increased over time by their ability to continually store and adapt to new data as it becomes available.

Disadvantages of Artificial Neural Networks

• Computational complexity: Training big neural networks with several layers and neurons takes a lot of processing power, which increases time and energy use.
• Large datasets are necessary for ANNs to be trained to provide good generalizations. Little data might cause overfitting or inadequate generalization.
• Overfitting: Particularly with noisy or little datasets, complex neural network designs are prone to overfit. This problem is lessened by regularization methods or dropout layers.
• Hyperparameter sensitivity: A lot of hyperparameters (learning rate, network architecture, activation functions) in ANNs require fine-tuning. Appropriate value selection can be difficult and have a big effect on model performance.
• Interpretability: Because ANNs are black-box systems, it might be challenging to figure out how they make their judgments. Particularly in fields where openness is essential, this lack of interpretability might be problematic.
• Deep neural network training may be laborious, especially on big datasets, occasionally needing a lot of time and computing power.
• Adversarial Attack Vulnerability: Adversarial attacks can cause false predictions from even minute and undetectable changes to input data.
• Data Dependency: ANNs mostly depend on the calibre and representativeness of the data they are trained on. Predictions from biased or unrepresentative models may be erroneous.
• gear Dependency: Complex neural networks may not be as accessible or practical for some applications or settings if they need specialised gear to develop and train.

Summary

With its linked nodes arranged into layers that process information, artificial neural networks (ANNs) resemble the brain's neural architecture. Through methods like backpropagation, these networks use iterative adjustments of internal parameters to learn from data. ANNs are excellent for classifying and regressing in a variety of fields including finance, natural language processing, and picture identification. Because of their flexibility and ability to manage complex tasks, they continue to be essential in advancing machine learning, robotics, and artificial intelligence (AI) even if they come with difficulties including computational complexity, data dependency, and interpretability problems.