2023 Fall ECE-5831 Neural Network & Pattern Recognition

Projects

Please note that accessing the datasets requires a umich email account.

Credit Card Fraud Detection

Summary:

This project focused on credit card fraud detection using a neural network trained on a dataset comprising transactions from September 2013. In a setting increasingly dominated by digital transactions, the model addressed the challenge of identifying and preventing fraudulent activities. Using techniques like Principal Components Analysis (PCA) for data dimension reduction, the model demonstrated promising results in a comprehensive classification report.

Simple GAN Project on MNIST and California Housing Data

Summary:

This project involves the application of a Simple Generative Adversarial Network (GAN) to two distinct datasets: the MNIST dataset of handwritten digits and the California Housing data. The project aims to explore the effectiveness of GANs in generating synthetic data that mimics the original datasets in terms of distribution and characteristics. The MNIST dataset
serves as a benchmark for the GAN model, while the extension to the California Housing data demonstrates the model’s adaptability to more complex, real-world datasets.

Enhancing mobility for people who are visually impaired
using Neural Networks

Summary:

This project employs an object detection system using the YOLOv8 algorithm, designed
to aid visually impaired individuals in navigating their environment. The Python
application, FinalCode.py, integrates advanced deep learning techniques with
real-time video processing. It utilizes a pretrained YOLO model for detecting footpaths
and obstacles in the camera’s view, running on a device with CUDA-enabled GPU for
efficient processing. The application, through cv2 and Pygame libraries, captures live
video feed and processes each frame to detect and annotate footpaths. A unique
feature of this system is its auditory feedback mechanism; based on the position of
detected objects relative to the camera’s center, it plays directional audio cues (left,
right, center) to guide the user. The application also displays the current frame rate
(FPS) on-screen, ensuring a smooth user experience.

Plant Disease Detection System using Deep Learning and CNN

Summary:

This article outlines the implementation of the Plant Disease web application, addressing the critical challenge of timely plant disease detection in agriculture. Recognizing the limitations of traditional manual inspection methods, the project leverages advanced Data Science techniques, particularly Convolutional Neural Networks (CNN), implemented using Keras. The focus is on streamlining and enhancing disease detection processes, mitigating economic burdens on farmers, and minimizing inaccuracies. By integrating image processing and deep learning, the project facilitates automated analysis of plant leaves, providing an efficient alternative to manual inspection. The inclusion of a user-friendly Flask application further enhances accessibility, allowing users to seamlessly upload images, receive real-time predictions, and visualize results. In essence, this project exemplifies the convergence of
agricultural demands with cutting-edge technologies, utilizing CNNs and a Flask application to offer an innovative solution for plant disease detection, contributing to global food security and economic prosperity.

Neural Network based Tumor Detection and Classification

Summary:

The choice of the most practical treatment technique depends on the early diagnosis and
categorization of brain tumors. In contrast to other malignancies where cancer stage is valued, brain tumors place a premium on categorization. Convolutional Neural Networks are used to create and deploy a system based on artificial intelligence (AI) for the identification and categorization of cancers using MRI data. This study would help physicians determine the type of tumor.

Deep Learning Based Weather Forecasting

Summary:

This project aims to experiment with two major deep learning models to perform the task of weather forecasting. The deep learning models experimented with are – ConvLSTM (Convolutional Long Short Term Memory) model and the classic UNet model.

Image Caption Generator using CNN and LSTM

Summary:

In this study, we undertake supervised learning to generate natural language captions based on a collection of images paired with their respective descriptions. Our approach involves constructing a neural network that combines CNN and LSTM models to extract image features and learn text sequences from the associated captions. We employ transfer learning by utilizing a pre-trained CNN model (Xception pre-trained on imagenet) to extract significant image features from the Flicker8K dataset. Textual features from the captions are obtained using GloVe embeddings, and the LSTM is utilized for language learning by processing these embeddings. Eventually, we merge the text and image features into a neural network, training it to generate captions for images it hasn’t previously encountered.

Text Summarization App

Summary:

In this text summarization project, I aimed to develop an effective and versatile system for condensing large bodies of text into concise summaries, catering to the increasing demand for efficient information processing. Leveraging advanced natural language processing techniques and pre-trained models, I employed the BART (Bidirectional and Auto-Regressive Transformers) architecture for conditional text generation. Key evaluation metrics such as BLEU and ROUGE scores were computed to assess the model’s summarization performance, providing insights into the system’s ability to generate accurate and coherent summaries. Additionally, the project featured a web-based interface that allowed users to interact with
the summarization system, showcasing its practical utility.

This project not only contributes to the field of natural language processing by showcasing the
effectiveness of transformer-based models in text summarization but also underscores the broader applications of such technologies in real-world scenarios. The combination of state-of-the-art models, rigorous evaluation methodologies, and user-friendly interfaces positions this project as a valuable tool for professionals and researchers seeking to streamline information consumption, making it particularly relevant in contexts where the quick extraction of salient information from extensive texts is crucial. The successful implementation and evaluation of this text summarization system highlight its potential for various domains, including journalism, research, and content curation, marking a step forward in advancing the capabilities of automated summarization technologies.

Along with the implementation of the text summarization app, I have tried to fine-tune the pre-trained model with CCN-Daily News which has 300k unique samples, but I couldn’t. My system which runs only the CPU is unable to tokenize the dataset and I have tried other datasets also; I found the same issue unable to tokenize the dataset which terminates indefinitely.

Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving

Summary:

The aim of this project is to implement a use of Large Language Models in the domain of
autonomous driving,and is based on a large-scale recent study which is a pioneer in the
advancement of autonomous systems, specifically addressing challenges in interpretability and decision-making. The LLM architecture combines numeric information with a pre-trained
language model to better understand driving situations. The study also developed a dataset with 160,000 question-answer pairs based on 10,000 driving scenarios, where the questions were generated by both a reinforcement learning agent and a teacher language model (GPT-3.5). However, in our study we worked with a subset of the dataset and have achieved an accuracy of 70%. Overall, the aim behind the implementation is to introduce explainability and clear interpretation of autonomous decisions.

End-To-End Driving in Orchard Farm

Summary:

End-to-End Driving has gained wide popularity among researchers both in Academia and
Industry. The idea is to give the raw input data into a neural network model, and it should predict the driving decisions regarding prediction, planning, and perception as one module. This technique has shown promising results in contrast to the traditional modular approach for making driving decisions. In this paper, we have used the End-to-End Driving paradigm in the agricultural domain. Our robot, named Scout, is able to navigate in the straight lane of an agricultural farm after the training. We used ROS for our simulation and Python programming language for the training and testing of the dataset. The paper contains a detailed description of the Simulation Environment and tools, Literature Review, our Data Collection Strategy, our proposed Model, and the results we accumulated during the training and testing. In the end of this paper, we also discussed our conclusion and the future work related to this project.

Human Action Recognition and Pose Detection

Summary:

This research explores the practical applications of computer vision through ConvLSTM and
LRCN models for human action recognition, revealing that ConvLSTM initially exhibits robustness but succumbs to overfitting, while LRCN consistently performs with high accuracy, precision, and recall. An external implementation integrates human action recognition and pose prediction. The study emphasizes the ethical responsibility of developing technology to assist those in need and discusses the transformative potential of computer vision. The literature review explores advancements in object and pose detection, highlighting ConvLSTM and LRCN architectures. The methodology details hyperparameters and design choices for ConvLSTM and LRCN models. Results indicate LRCN’s reduced susceptibility to overfitting
compared to ConvLSTM. Testing on unseen data shows challenges, and an external implementation combines human action recognition and pose prediction. Future improvements involve exploring sophisticated architectures, data augmentation, transfer learning, hyperparameter tuning, and addressing class imbalances for robust model deployment.

Feature based Recommendation System Using Ecommerce Data

Summary:

This project focuses on creating a feature-based recommendation model for an e-commerce data using user interaction and product data. Through careful preprocessing, feature engineering, and model development, the aim is to enhance user experience & product recommendation system by providing personalized product recommendations using feature based recommendation model. The model was trained, validated, and integrated into the UI platformusing Flask API, with ongoing monitoring and potential future improvements. The
expected outcomes include a good precision value and top 5 products being recommended on selecting different products.

PLANT DISEASE DETECTION

Summary:

The study introduces a potent plant disease detection system merging CNN-based image
feature extraction with machine learning. It utilizes the Plant-Village Dataset, employing
preprocessing techniques like color segmentation and feature extraction. Seven machine
learning model states the evaluations, achieving 95% accuracy in identifying unhealthy plants. By combining traditional ML methods with CNN-based image processing, the system offers
precision and effectiveness in addressing agricultural challenges.

SKIN CANCER DETECTION USING CONVOLUTIONAL NEURAL NETWORKS (CNN)

SUMMARY:

In our project on skin cancer detec[on using Convolu[onal Neural Networks (CNNs), we aimed
to classify melanoma from a diverse dataset of skin lesions. We experimented with three CNN
architectures, each introducing diﬀerent features such as basic layers, data augmenta[on, and
batch normaliza[on. The models were trained and validated, achieving notable accuracy
metrics. The third architecture, incorpora[ng batch normaliza[on, demonstrated the highest
accuracy of 96.6% in training and 86.3% in valida[on. While limita[ons in computa[onal
resources constrained dataset size, our work provides a founda[on for future enhancements,
sugges[ng the explora[on of diverse CNN architectures and larger datasets for op[mal skin
cancer diagnosis. The confusion matrix analysis oﬀers insights into areas for poten[al model
improvement, emphasizing the importance of reﬁning the classiﬁca[on of certain skin lesion
classes.

Unpaired Image-to-Image Style Translation using CycleGAN

Summary:

In this project, our primary objective is to replicate and extend the findings of two research papers focusing on unpaired image-to-image translation using CycleGAN. The first aspect of our implementation involves the utilization of CycleGAN for the translation task which produces a Monet-styled image for any image given to the model. CycleGAN is a powerful deep-learning model for unsupervised image translation, enabling the transformation of images from one domain to another without the need for paired data. Additionally, we are incorporating another model inspired by the second research paper, which leverages
differentiable data augmentation and employs callbacks for dynamic adjustments during training. This model incorporates strategies such as updating learning rates, and weights to enhance the overall training process. The optimization process is specifically designed to achieve a lower cycle consistency loss, thereby improving the quality of translated images.
Our project evaluates the performance using key metrics, including generator loss, cycle consistency loss, and discriminator loss. These metrics serve as crucial indicators of the model’s effectiveness in capturing and reproducing stylistic features in image translation.

Credit Card Fraud Detection

Summary:

Using synthetic data generated by IBM and freely available on Kaggle, we trained a model to
predict whether a credit card transaction is fraudulent or nonfraudulent based on
information about the user, the card used, and the transaction itself. The model was
developed in tensorflow and the data was loaded and prepared using pandas and numpy.
The code was written in Python packages and Jupyther Notebook files, producing pickle’d
files containing the prepared data, as well as one .keras file containing the trained model. Our
model showed heavy bias towards negatives in the end, but previous iterations of the same
model using the same parameters for the training and validation set building showcased high
accuracy when evaluated with a balanced validation set. Our conclusion is that the validation
and training data contained unidentified patterns or ratios affected the quality of the
randomly sampled fraudulent transactions.

Computer Vision-Based Intelligent ADAS for Enhanced Road Safety: SignWatch System

Summary:

This research project addresses the pressing road safety issue by introducing a comprehensive computer vision-based system designed to enhance driver awareness and compliance with traffic regulations. Beyond the primary function of real-time traffic sign classification, the system incorporates multi-language audio descriptions, instantaneous alerts, and valuable information delivery to provide holistic support to drivers. Additionally, the
integration of dynamic weather data aims to equip drivers with essential information for navigating diverse weather conditions safely. This amalgamation of functionalities underscores the project’s commitment to creating a robust and user-centric road safety solution, utilizing innovative technology to contribute to a safer and more informed driving environment. The project emphasizes the need to transcend traditional approaches and underscores the broader implications of individual efforts in advancing road safety through technology-driven initiatives.

State of Charge (SOC) and State of Health (SOH) Estimation in Second-Life Batteries (SLB) Using Artificial Neural Networks (ANN)

Summary:

In our recent project, we embarked on an ambitious endeavor to predict the State of Charge (SoC) and State of Health (SoH) of Solid Lithium Batteries (SLBs), a subject that has garnered significant attention in contemporary research. Our approach involved utilizing PybaMM, a powerful tool for simulating the behavior of SLBs under various conditions. Despite facing challenges in extracting appropriate data for our project, the experience was highly educational, providing us with deep insights into the intricacies of different SLB types. This process also highlighted the necessity of finding efficient methods to reduce the time required for extracting models from SLB data. Consequently, we shifted our focus towards predicting the SoC of real Electric Vehicles (EVs), leveraging data that had been meticulously collected.
For this purpose, we utilized a dataset from Kaggle, implementing a neural network architecture comprising three hidden layers. The outcome of our work was encouraging, as the results demonstrated that our model was capable of making predictions with satisfactory accuracy, marking a significant step forward in our understanding and application of battery technology.

Detection of Dementia using Clock Drawing Tests

Summary:

This project deals with Identifying patients with Dementia using Clock Drawings (Clock showing 11:10). It is a standard test to identify the quality of Clock drawings and rate the extent of Dementia od patients between 0(Lowest – Severe Dementia) to 5 (Normal Patient).
Almost 6 million individuals in the US alone are affected by Alzheimer’s and Dementia related
diseases. Clock Drawing Test (CDT) is a cognitive assessment to detect it. In this project, I have made use of Transformer models like Vision Transformers, ResNet and other Deep Learning models to perform automated scoring on the Clock Drawing Tests. This approach can eliminate the manual coding performed today. The Vision Transformers provided the best accuracy of around 78% on the Clock Drawing images obtained from NHATS website. The training process involved around 22500 images and the Testing dataset had about 2250 images. The project also involves a Model performance comparison and multiple Ensemble approaches to try and improve the performance of the model.

Plant Disease Detection System using Deep Learning and CNN

Summary:

Age, Gender, and Ethnicity Prediction using Deep Learning

Summary:

Our project has created a deep learning model that can predict a person’s age, gender, and
ethnicity from their facial image, using a specialized multi-task learning CNN and the
comprehensive UTKFace dataset. Aimed at enhancing the precision of facial recognition
technology, the project also emphasizes ethical AI by ensuring diversity and fairness in training.
The model’s successful application in both static image analysis and real-time prediction
showcases its potential for wide-ranging uses, from personalizing digital experiences to
strengthening security measures. Acknowledging the need for continuous improvement, the
project sets the stage for future advancements in creating more inclusive and unbiased AI
systems.

Neural Network Based Breast Cancer Diagnostics

Summary

We have developed a breast cancer diagnostic system that relies on neural networks. The primary objective of this project is to create an efficient model capable of accurately classifying breast masses as either malignant or benign by utilizing patient characteristics and tumor features.

Neural Network Based Identification of a Non-Linear Electronic System

Summary:

Designing a controller for an unknown system can be very difficult task. Furthermore, typical
adaptive controllers that are capable of controlling an unknown system can be very heavy and difficult to implement in an embedded real time environment. It has been proposed that neural networks are a possible solution to perform adaptive control without requiring the same amount of computational resources needed for typical adaptive controllers. In order to operate in a real time environment, the control system must be able to accurately predict the output of the unknown system in response to a particular control signal. This predictor can also take the form of a neural network that has been trained to emulate the system. This project explores the ability of some minimal neural network configuration to accurately predict the output of six selected electrical circuits.

HUMAN POSE ESTIMATION USING DEEP LEARNING

Summary:

This project represents a significant advance in human estimation by applying deep learning using the Pose Estimation With MobileNet model on the COCO dataset. This approach involves careful preprocessing of data, structured modeling, and effective training methods powered by Python scripts. The selected models show a balance between performance and accuracy. The project not only addresses the challenges of optimizing preliminary data and model, but also explores the complexity of the OpenPose process. The results show that the model has good knowledge of accurately detecting and tracking the key points of the human body, with visual thermal imaging as visual evidence. Integration of the decision using the Intel OpenVINO suite of tools improves real-world implementation. Overall, this project provides good results and a promising perspective for the continued improvement of human prediction using deep learning techniques.

Traffic Sign Recognition

Summary:

The project’s main goal is to improve traffic management, enable driver assistance
technologies, support road safety, and advance autonomous vehicle technology by utilizing
computer vision technology to create a real-time traffic sign detection system. The project’s
history, related work, dataset preprocessing, model development, and preliminary findings are
all covered in the report and presentation. With the use of Convolutional Neural Networks
(CNNs), a machine learning technique, the methodology recognizes traffic signs accurately in
a variety of environmental scenarios. The Traffic Sign Recognition project is a critical response
to the growing problems in urban transportation, especially in light of the development of
advanced driver assistance systems and autonomous vehicles.

Medical Image Detection and Classification of Brain Tumors

Summary:

My final project for ECE 5831 was based on medical image detection and more specifically classifying different kinds of brain tumors and also detecting whether a tumor was even present at all. I found this project very eye opening and it allowed me to experience the true hands-on uses of machine learning that were different than just regular class ssignments.
I had two different data sets from Kaggle of MRI medical image scans. I also trained models for binary classification of tumors and also a model for multiclass classification of tumor (Glioma, Meningioma, Pituitary, and no tumor). I ran test images through the models and had high accuracy rates. In the future I would like to expand this to other medical images and scenarios that are not just limited to brain tumors.

Email Classifier

Summary:

Email communication has become an integral part of our lives, but with its widespread use comes the inevitable challenge of spam emails. The increasing volume of spam emails each year poses a serious threat to our digital society. This project aims to develop a spam and ham email classifier using the Naive Bayes model. This algorithm revolves around a content-based approach, consisting of two main phases: training and classification. The key player here is the keyword-based corpus, constructed meticulously from the content of emails. This corpus serves as the backbone for our spam detection mechanism. With over 6,000 emails in the training and testing datasets combined, the experimental results showcase an impressive 94.4% accuracy.

Face Recognition Security System

Summary:

The project explores the fields of pattern recognition and computer vision to leverage the laptop camera to detect and recognize faces, allowing access only to authorized users. A key aspect of this system is its ability to trigger an alarm mechanism in the event of unauthorized
access, an innovative warning system has been created in the form of an email notification that immediately notify the user of unauthorized access attempts.

M	T	W	T	F	S	S
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30

2023 Fall ECE-5831 Neural Network & Pattern Recognition

Projects

Credit Card Fraud Detection

Summary:

Links:

Simple GAN Project on MNIST and California Housing Data

Summary:

Links:

Enhancing mobility for people who are visually impairedusing Neural Networks

Summary:

Links:

Plant Disease Detection System using Deep Learning and CNN

Summary:

Links:

Neural Network based Tumor Detection and Classification

Summary:

Links:

Deep Learning Based Weather Forecasting

Summary:

Links:

Image Caption Generator using CNN and LSTM

Summary:

Links:

Text Summarization App

Summary:

Links:

Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving

Summary:

Links:

End-To-End Driving in Orchard Farm

Summary:

Links:

Human Action Recognition and Pose Detection

Summary:

Links:

Feature based Recommendation System Using Ecommerce Data

Summary:

Links:

PLANT DISEASE DETECTION

Summary:

Links:

SKIN CANCER DETECTION USING CONVOLUTIONAL NEURAL NETWORKS (CNN)

SUMMARY:

Links

Unpaired Image-to-Image Style Translation using CycleGAN

Summary:

Links:

Credit Card Fraud Detection

Summary:

Links:

Computer Vision-Based Intelligent ADAS for Enhanced Road Safety: SignWatch System

Summary:

Links:

State of Charge (SOC) and State of Health (SOH) Estimation in Second-Life Batteries (SLB) Using Artificial Neural Networks (ANN)

Summary:

Links:

Detection of Dementia using Clock Drawing Tests

Summary:

Links:

Plant Disease Detection System using Deep Learning and CNN

Summary:

Links:

Age, Gender, and Ethnicity Prediction using Deep Learning

Summary:

Links:

Neural Network Based Breast Cancer Diagnostics

Summary

Links:

Neural Network Based Identification of a Non-Linear Electronic System

Summary:

Links:

HUMAN POSE ESTIMATION USING DEEP LEARNING

Summary:

Links:

Traffic Sign Recognition

Summary:

Links:

Medical Image Detection and Classification of Brain Tumors

Summary:

Links:

Enhancing mobility for people who are visually impaired
using Neural Networks