Electrical & Computer Engineering Department Theses and Dissertations

Permanent URI for this collection

https://hdl.handle.net/1805/2087

Information about the Purdue School of Engineering and Technology Graduate Degree Programs available at IUPUI can be found at: http://www.engr.iupui.edu/academics.shtml

Browse

Now showing 1 - 10 of 198

Squeeze and Excite Residual Capsule Network for Embedded Edge Devices
(2022-08) Naqvi, Sami; El-Sharkawy, Mohamed; King, Brian; Rizkalla, Maher
During recent years, the field of computer vision has evolved rapidly. Convolutional Neural Networks (CNNs) have become the chosen default for implementing computer vision tasks. The popularity is based on how the CNNs have successfully performed the well-known computer vision tasks such as image annotation, instance segmentation, and others with promising outcomes. However, CNNs have their caveats and need further research to turn them into reliable machine learning algorithms. The disadvantages of CNNs become more evident as the approach to breaking down an input image becomes apparent. Convolutional neural networks group blobs of pixels to identify objects in a given image. Such a technique makes CNNs incapable of breaking down the input images into sub-parts, which could distinguish the orientation and transformation of objects and their parts. The functions in a CNN are competent at learning only the shift-invariant features of the object in an image. The discussed limitations provides researchers and developers a purpose for further enhancing an effective algorithm for computer vision. The opportunity to improve is explored by several distinct approaches, each tackling a unique set of issues in the convolutional neural network’s architecture. The Capsule Network (CapsNet) which brings an innovative approach to resolve issues pertaining to affine transformations by sharing transformation matrices between the different levels of capsules. While, the Residual Network (ResNet) introduced skip connections which allows deeper networks to be more powerful and solves vanishing gradient problem. The motivation of these fusion of these advantageous ideas of CapsNet and ResNet with Squeeze and Excite (SE) Block from Squeeze and Excite Network, this research work presents SE-Residual Capsule Network (SE-RCN), an efficient neural network model. The proposed model, replaces the traditional convolutional layer of CapsNet with skip connections and SE Block to lower the complexity of the CapsNet. The performance of the model is demonstrated on the well known datasets like MNIST and CIFAR-10 and a substantial reduction in the number of training parameters is observed in comparison to similar neural networks. The proposed SE-RCN produces 6.37 Million parameters with an accuracy of 99.71% on the MNIST dataset and on CIFAR-10 dataset it produces 10.55 Million parameters with 83.86% accuracy.
DFIG-Based Split-Shaft Wind Energy Conversion Systems
(2022-08) Akbari, Rasoul; Izadian, Afshin; Dos Santos, Euzeli; King, Brian; Weissbach, Robert
In this research, a Split-Shaft Wind Energy Conversion System (SS-WECS) is investigated to improve the performance and cost of the system and reduce the wind power uncertainty influences on the power grid. This system utilizes a lightweight Hydraulic Transmission System (HTS) instead of the traditional gearbox and uses a Doubly-Fed Induction Generator (DFIG) instead of a synchronous generator. This type of wind turbine provides several benefits, including decoupling the shaft speed controls at the turbine and the generator. Hence, maintaining the generator’s frequency and seeking maximum power point can be accomplished independently. The frequency control relies on the mechanical torque adjustment on the hydraulic motor that is coupled with the generator. This research provides modeling of an SS-WECS to show its dependence on mechanical torque and a control technique to realize the mechanical torque adjustments utilizing a Doubly-Fed Induction Generator (DFIG). To this end, a vector control technique is employed, and the generator electrical torque is controlled to adjust the frequency while the wind turbine dynamics influence the system operation. The results demonstrate that the generator’s frequency is maintained under any wind speed experienced at the turbine. Next, to reduce the size of power converters required for controlling DFIG, this research introduces a control technique that allows achieving MPPT in a narrow window of generator speed in an SS-WECS. Consequently, the size of the power converters is reduced significantly. The proposed configuration is investigated by analytical calculations and simulations to demonstrate the reduced size of the converter and dynamic performance of the power generation. Furthermore, a new configuration is proposed to eliminate the Grid- Side Converter (GSC). This configuration employs only a reduced-size Rotor-Side Converter (RSC) in tandem with a supercapacitor. This is accomplished by employing the hydraulic transmission system (HTS) as a continuously variable and shaft decoupling transmission unit. In this configuration, the speed of the DFIG is controlled by the RSC to regulate the supercapacitor voltage without GSC. The proposed system is investigated and simulated in MATLAB Simulink at various wind speeds to validate the results. Next, to reduce the wind power uncertainty, this research introduces an SS-WECS where the system’s inertia is adjusted to store the energy. Accordingly, a flywheel is mechanically coupled with the rotor of the DFIG. Employing the HTS in such a configuration allows the turbine controller to track the point of maximum power (MPPT) while the generator controller can adjust the generator speed. As a result, the flywheel, which is directly connected to the shaft of the generator, can be charged and discharged by controlling the generator speed. In this process, the flywheel energy can be used to modify the electric power generation of the generator on-demand. This improves the quality of injected power to the grid. Furthermore, the structure of the flywheel energy storage is simplified by removing its dedicated motor/generator and the power electronics driver. Two separate supervisory controllers are developed using fuzzy logic regulators to generate a real-time output power reference. Furthermore, small-signal models are developed to analyze and improve the MPPT controller. Extensive simulation results demonstrate the feasibility of such a system and its improved quality of power generation. Next, an integrated Hybrid Energy Storage System (HESS) is developed to support the new DFIG excitation system in the SS-WECS. The goal is to improve the power quality while significantly reducing the generator excitation power rating and component counts. Therefore, the rotor excitation circuit is modified to add the storage to its DC link directly. In this configuration, the output power fluctuation is attenuated solely by utilizing the RSC, making it self-sufficient from the grid connection. The storage characteristics are identified based on several system design parameters, including the system inertia, inverter capacity, and energy storage capacity. The obtained power generation characteristics suggest an energy storage system as a mix of fast-acting types and a high energy capacity with moderate acting time. Then, a feedback controller is designed to maintain the charge in the storage within the required limits. Additionally, an adaptive model-predictive controller is developed to reduce power generation fluctuations. The proposed system is investigated and simulated in MATLAB Simulink at various wind speeds to validate the results and demonstrate the system’s dynamic performance. It is shown that the system’s inertia is critical to damping the high-frequency oscillations of the wind power fluctuations. Then, an optimization approach using the Response Surface Method (RSM) is conducted to minimize the annualized cost of the Hybrid Energy Storage System (HESS); consisting of a flywheel, supercapacitor, and battery. The goal is to smooth out the output power fluctuations by the optimal size of the HESS. Thus, a 1.5 MW hydraulic wind turbine is simulated, and the HESS is configured and optimized. The direct connection of the flywheel allows reaching a suitable level of smoothness at a reasonable cost. The proposed configuration is compared with the conventional storage, and the results demonstrate that the proposed integrated HESS can decrease the annualized storage cost by 71 %. Finally, this research investigates the effects of the reduced-size RSC on the Low Voltage Ride Through (LVRT) capabilities required from all wind turbines. One of the significant achievements of an SS-WECS is the reduced size excitation circuit. The grid side converter is eliminated, and the size of the rotor side converter (RSC) can be safely reduced to a fraction of a full-size excitation. Therefore, this low-power-rated converter operates at low voltage and handles the regular operation well. However, the fault conditions may expose conditions on the converter and push it to its limits. Therefore, four different protection circuits are employed, and their effects are investigated and compared to evaluate their performance. These four protection circuits include the active crowbar, active crowbar along a resistorinductor circuit (C-RL), series dynamic resistor (SDR), and new-bridge fault current limiter (NBFCL). The wind turbine controllers are also adapted to reduce the impact of the fault on the power electronic converters. One of the effective methods is to store the excess energy in the generator’s rotor. Finally, the proposed LVRT strategies are simulated in MATLAB Simulink to validate the results and demonstrate their effectiveness and functionality.
Efficient Wearable Big Data Harnessing and Mining with Deep Intelligence
(2022-08) Basile, Elijah James; Zhang, Qingxue; King, Brian; Schubert, Peter
Wearable devices and their ubiquitous use and deployment across multiple areas of health provide key insights in patient and individual status via big data through sensor capture at key parts of the individual’s body. While small and low cost, their limitations rest in their computational and battery capacity. One key use of wearables has been in individual activity capture. For accelerometer and gyroscope data, oscillatory patterns exist between daily activities that users may perform. By leveraging spatial and temporal learning via CNN and LSTM layers to capture both the intra and inter-oscillatory patterns that appear during these activities, we deployed data sparsification via autoencoders to extract the key topological properties from the data and transmit via BLE that compressed data to a central device for later decoding and analysis. Several autoencoder designs were developed to determine the principles of system design that compared encoding overhead on the sensor device with signal reconstruction accuracy. By leveraging asymmetric autoencoder design, we were able to offshore much of the computational and power cost of signal reconstruction from the wearable to the central devices, while still providing robust reconstruction accuracy at several compression efficiencies. Via our high-precision Bluetooth voltmeter, the integrated sparsified data transmission configuration was tested for all quantization and compression efficiencies, generating lower power consumption to the setup without data sparsification for all autoencoder configurations. Human activity recognition (HAR) is a key facet of lifestyle and health monitoring. Effective HAR classification mechanisms and tools can provide healthcare professionals, patients, and individuals key insights into activity levels and behaviors without the intrusive use of human or camera observation. We leverage both spatial and temporal learning mechanisms via CNN and LSTM integrated architectures to derive an optimal classification architecture that provides robust classification performance for raw activity inputs and determine that a LSTMCNN utilizing a stacked-bidirectional LSTM layer provides superior classification performance to the CNNLSTM (also utilizing a stacked-bidirectional LSTM) at all input widths. All inertial data classification frameworks are based off sensor data drawn from wearable devices placed at key sections of the body. With the limitation of wearable devices being a lack of computational and battery power, data compression techniques to limit the quantity of transmitted data and reduce the on-board power consumption have been employed. While this compression methodology has been shown to reduce overall device power consumption, this comes at a cost of more-or-less information loss in the reconstructed signals. By employing an asymmetric autoencoder design and training the LSTMCNN classifier with the reconstructed inputs, we minimized the classification performance degradation due to the wearable signal reconstruction error The classifier is further trained on the autoencoder for several input widths and with quantized and unquantized models. The performance for the classifier trained on reconstructed data ranged between 93.0\% and 86.5\% accuracy dependent on input width and autoencoder quantization, showing promising potential of deep learning with wearable sparsification.
Comparing Pso-Based Clustering Over Contextual Vector Embeddings to Modern Topic Modeling
(2022-05) Miles, Samuel; Ben Miled, Zina; Salama, Paul; El-Sharkawy, Mohamed
Efficient topic modeling is needed to support applications that aim at identifying main themes from a collection of documents. In this thesis, a reduced vector embedding representation and particle swarm optimization (PSO) are combined to develop a topic modeling strategy that is able to identify representative themes from a large collection of documents. Documents are encoded using a reduced, contextual vector embedding from a general-purpose pre-trained language model (sBERT). A modified PSO algorithm (pPSO) that tracks particle fitness on a dimension-by-dimension basis is then applied to these embeddings to create clusters of related documents. The proposed methodology is demonstrated on three datasets across different domains. The first dataset consists of posts from the online health forum r/Cancer. The second dataset is a collection of NY Times abstracts and is used to compare
Multi-Source and Source-Private Cross-Domain Learning For Visual Recognition
(2022-05) Peng, Qucheng; Li, Lingxi; Ding, Zhengming; Zhang, Qingxue; King, Brian
Domain adaptation is one of the hottest directions in solving annotation insufficiency problem of deep learning. General domain adaptation is not consistent with the practical scenarios in the industry. In this thesis, we focus on two concerns as below. First is that labeled data are generally collected from multiple domains. In other words, multi-source adaptation is a more common situation. Simply extending these single-source approaches to the multi-source cases could cause sub-optimal inference, so specialized multi-source adaptation methods are essential. The main challenge in the multi-source scenario is a more complex divergence situation. Not only the divergence between target and each source plays a role, but the divergences among distinct sources matter as well. However, the significance of maintaining consistency among multiple sources didn't gain enough attention in previous work. In this thesis, we propose an Enhanced Consistency Multi-Source Adaptation (EC-MSA) framework to address it from three perspectives. First, we mitigate feature-level discrepancy by cross-domain conditional alignment, narrowing the divergence between each source and target domain class-wisely. Second, we enhance multi-source consistency via dual mix-up, diminishing the disagreements among different sources. Third, we deploy a target distilling mechanism to handle the uncertainty of target prediction, aiming to provide high-quality pseudo-labeled target samples to benefit the previous two aspects. Extensive experiments are conducted on several common benchmark datasets and demonstrate that our model outperforms the state-of-the-art methods. Second is that data privacy and security is necessary in practice. That is, we hope to keep the raw data stored locally while can still obtain a satisfied model. In such a case, the risk of data leakage greatly decreases. Therefore, it is natural for us to combine the federated learning paradigm with domain adaptation. Under the source-private setting, the main challenge for us is to expose information from the source domain to the target domain while make sure that the communication process is safe enough. In this thesis, we propose a method named Fourier Transform-Assisted Federated Domain Adaptation (FTA-FDA) to alleviate the difficulties in two ways. We apply Fast Fourier Transform to the raw data and transfer only the amplitude spectra during the communication. Then frequency space interpolations between these two domains are conducted, minimizing the discrepancies while ensuring the contact of them and keeping raw data safe. What's more, we make prototype alignments by using the model weights together with target features, trying to reduce the discrepancy in the class level. Experiments on Office-31 demonstrate the effectiveness and competitiveness of our approach, and further analyses prove that our algorithm can help protect privacy and security.
Sensor Fusion in Neural Networks For Object Detection
(2022-05) Prasanna, Sheetal; El-Sharkawy, Mohamed; King, Brian; Rizkalla, Maher
Object detection is an increasingly popular tool used in many fields, especially in the development of autonomous vehicles. The task of object detections involves the localization of objects in an image, constructing a bounding box to determine the presence and location of the object, and classifying each object into its appropriate class. Object detection applications are commonly implemented using convolutional neural networks along with the construction of feature pyramid networks to extract data. Another commonly used technique in the automotive industry is sensor fusion. Each automotive sensor – camera, radar, and lidar – have their own advantages and disadvantages. Fusing two or more sensors together and using the combined information is a popular method of balancing the strengths and weakness of each independent sensor. Together, using sensor fusion within an object detection network has been found to be an effective method of obtaining accurate models. Accurate detections and classifications of images is a vital step in the development of autonomous vehicles or self-driving cars. Many studies have proposed methods to improve neural networks or object detection networks. Some of these techniques involve data augmentation and hyperparameter optimization. This thesis achieves the goal of improving a camera and radar fusion network by implementing various techniques within these areas. Additionally, a novel idea of integrating a third sensor, the lidar, into an existing camera and radar fusion network is explored in this research work. The models were trained on the Nuscenes dataset, one of the biggest automotive datasets available today. Using the concepts of augmentation, hyperparameter optimization, sensor fusion, and annotation filters, the CRF-Net was trained to achieve an accuracy score that was 69.13% higher than the baseline.
The Design of an Oncology Knowledge Base from an Online Health Forum
(2022-05) Ramadan, Omar; Ben Miled, Zina; Salama, Paul; Dos Santos, Euzeli Cipriano
Knowledge base completion is an important task that allows scientists to reason over knowledge bases and discover new facts. In this thesis, a patient-centric knowledge base is designed and constructed using medical entities and relations extracted from the health forum r/cancer. The knowledge base stores information in binary relation triplets. It is enhanced with an is-a relation that is able to represent the hierarchical relationship between different medical entities. An enhanced Neural Tensor Network that utilizes the frequency of occurrence of relation triplets in the dataset is then developed to infer new facts from the enhanced knowledge base. The results show that when the enhanced inference model uses the enhanced knowledge base, a higher accuracy (73.2 %) and recall@10 (35.4%) are obtained. In addition, this thesis describes a methodology for knowledge base and associated inference model design that can be applied to other chronic diseases.
Efficientnext: Efficientnet For Embedded Systems
(2022-05) Deokar, Abhishek; El-Sharkawy, Mohamed; King, Brian; Rizkalla, Maher
Convolutional Neural Networks have come a long way since AlexNet. Each year the limits of the state of the art are being pushed to new levels. EfficientNet pushed the performance metrics to a new high and EfficientNetV2 even more so. Even so, architectures for mobile applications can benefit from improved accuracy and reduced model footprint. The classic Inverted Residual block has been the foundation upon which most mobile networks seek to improve. EfficientNet architecture is built using the same Inverted Residual block. In this thesis we experiment with Harmonious Bottlenecks in place of the Inverted Residuals to observe a reduction in the number of parameters and improvement in accuracy. The designed network is then deployed on the NXP i.MX 8M Mini board for Image classification.
Multi-Class Vocation Identification for Heavy Duty Vehicles
(2021-12) Yadav, Varun; Ben-Miled, Zina; Dos Santos, Euzeli; Salama, Paul
Understanding the operating profile of different heavy-duty vehicles is needed by parts manufacturers for improved configuration and better future design of the parts. This study investigates the use of a tournament classification approach for both vocation and fleet identi- fication. The proposed approach is implemented using four different classification techniques, namely, K-Means, Expectation Maximization, Particle Swarm Optimization, and Support Vector Machines. Vocations classifiers are developed and tested for six different vocations ranging from coach buses to rail inspection vehicles. Operational field data are obtained from a number of vehicles for each vocation and aggregated over a pre-set distance that varies according to the data collection rate. In addition, fleet classifiers are implemented for five fleets from the coach bus vocation using a similar approach. The results indicate that both vocation and fleet identification are possible with a high level of accuracy. The macro average precision and recall of the SVM vocation classifier are approximately 85%. This result was achieved despite the fact that each vocation consisted of multiple fleets. The macro average precision and recall of the coach bus fleet classifier are approximately 77% even though some fleets had similar operating profiles. These results suggest that the proposed classifier can help support vocation and fleet identification in practice.
Global Translation of Machine Learning Models to Interpretable Models
(2021-12) Almerri, Mohammad; Ben Miled, Zina; Christopher, Lauren; Salama, Paul
The widespread and growing usage of machine learning models, especially in highly critical areas such as law, predicate the need for interpretable models. Models that cannot be audited are vulnerable to inheriting biases from the dataset. Even locally interpretable models are vulnerable to adversarial attack. To address this issue a new methodology is proposed to translate any existing machine learning model into a globally interpretable one. This methodology, MTRE-PAN, is designed as a hybrid SVM-decision tree model and leverages the interpretability of linear hyperplanes. MTRE-PAN uses this hybrid model to create polygons that act as intermediates for the decision boundary. MTRE-PAN is compared to a previously proposed model, TRE-PAN, on three non-synthetic datasets: Abalone, Census and Diabetes data. TRE-PAN translates a machine learning model to a 2-3 decision tree in order to provide global interpretability for the target model. The datasets are each used to train a Neural Network that represents the non-interpretable model. For all target models, the results show that MTRE-PAN generates interpretable decision trees that have a lower number of leaves and higher parity compared to TRE-PAN.

Browse

Recent Submissions