Power Bi Azure Data

PROFESSIONAL SUMMARY:

**+ years of experience in Data Analysis, Design, Implementation, Administration and support of Business Intelligence using SQL Server SSIS, SSAS and SSRS.

Experienced in building and architecting multiple Data pipelines, end to end ETL and ELT process for Data ingestion and transformation.

Hands-on experience in using other AWS (Amazon Web Services) like AWS S3, VPC, EC2, Auto scaling, Redshift, Dynamo DB, Route53, RDS, Glacier and AWS EMR.

Expertise in other AWS services such as AWS Glue, Athena and AWS IAM.

Proficient in utilizing a wide array of Azure data services such as Azure SQL Database, Azure Data Lake Storage and Azure Blob Storage to design and implement scalable data solutions.

Expertise in designing and implementing real-time data processing solutions using Azure Data Bricks and Azure Stream Analytics, facilitating rapid insights extraction from streaming data sources.

Hands-on experience on Google Cloud Platform (GCP) in all the big data products Big Query, Cloud Data Proc, Google Cloud Storage and Composer.

Created documentation for Cognos reports, including user guides and technical specifications, to facilitate knowledge sharing and support.

Proactively identified and resolved data discrepancies and inconsistencies in Cognos reports to maintain data accuracy.

Stayed abreast of industry best practices and emerging trends in Cognos reporting to continuously enhance reporting solutions.

Acted as a liaison between business users and IT teams to address feedback and enhance the usability of Cognos reports.

In-depth knowledge of Informatica Cloud for hybrid and cloud-based data integration, enabling seamless connectivity between on-premises and cloud environments.

Strong Programming/Scripting skills in Python, R, SQL and Scala.

Expertise in statistical programming languages like Python and R including Big-Data technologies like Hadoop, HDFS, Spark and Hive.

Proficient in database migration and upgrade processes, ensuring smooth transitions between different SQL server versions.

Proficient in implementing data partitioning and distribution strategies for improved performance in NoSQL distributed databases.

Skilled in MongoDB sharding and replication strategies to enhance data distribution, reliability, and disaster recovery capabilities.

Experienced in database version control and migration tools for efficient management of PostgreSQL database changes.

Good understanding on Apache Zookeeper and Kafka for monitoring and managing Hadoop jobs and using Cloudera CDH4 and CDH5 for monitoring and managing Hadoop clusters.

Hands-on experience in data ingestion, extraction, and transformation processes, utilizing tools like Apache NiFi and Apache Sqoop to streamline data workflows.

Successfully deployed StreamSets to streamline the ingestion of structured and unstructured data from diverse sources such as AWS S3 and Azure Blob Storage, enabling efficient data movement into downstream processing systems.

Configured StreamSets with security features such as encryption, access controls, and data masking to ensure compliance with regulatory requirements and safeguard sensitive data across the data lifecycle.

Fine-tuned StreamSets pipelines for optimal performance and scalability, optimizing resource utilization and throughput to meet demanding SLAs and business requirements.

Good working experience on Spark (Spark Core Component, Spark SQL, Spark Streaming, Spark MLLib, Spark GraphX) with Scala and Kafka.

Hands-on experience in implementing data partitioning and clustering strategies in Snowflake, optimizing storage and query performance for large-scale datasets.

Extensive background in designing and implementing data warehouses, incorporating Tableau and Power BI as integral components for creating centralized, accessible, and scalable reporting solutions.

Proficient in embedding Tableau visualizations into web applications and portals, enhancing data accessibility for non-technical users.

Expertise in Power BI paginated report design, optimizing layouts for print and ensuring consistency across different devices.

Imported data from SQL Server DB, Azure SQL DB to Power BI to generate reports.

Created Dax Queries to generated compute/measured columns in Power BI.

Generated computed tables in Power BI by using Dax

Used various sources to pull data into Power BI such as Sql Server, SQL Azure etc.

Worked on all kind of reports such as Yearly, Quarterly, Monthly, and Daily.

Published Power BI Reports in the required originations and Made Power BI Dashboards available in Web clients and mobile apps

Explore data in a variety of ways and across multiple visualizations using Power BI

Used Power BI Gateways to keep the dashboards and reports up to date.

Published reports and dashboards using Power BI.

Proficient in version control using Git and GitHub, ensuring seamless collaboration and code integrity within a data engineering environment.

Adept at utilizing Docker in CI/CD pipelines to automate the testing and deployment of data engineering solutions, reducing time-to-delivery and improving overall system reliability.

CERTIFICATIONS:

Microsoft Certified: Power BI Data Analyst Associate

AWS Certified Data Engineer

Microsoft Certified: Azure Data Engineer

TECHNICAL SKILLS:

Programming Languages

Python, R, SQL, PL/SQL, Scala, Unix Shell Scripting, T-SQL

Data Integration Tools

Informatica Cloud, Apache NiFi, Apache Sqoop, AWS Glue, Talend, Teradata

Big Data Technologies

Hadoop, HDFS, Spark, Hive, Pig, Impala, Kafka, Flink

Clouds

AWS, Azure

Databases

SQL Server Analysis Services, MYSQL, Oracle, PostgreSQL, MongoDB, Snowflake

Data Warehousing

Microsoft Power BI, Snowflake, Tableau Server, Power BI Paginated Reports

Reporting Tools

Power BI, Tableau, SQL Server Reporting Services (SSRS), Crystal Reports, SQL Server Management Studio, Power BI Report Builder.

ETL Tools

Teradata, Talend, AWS Glue, Informatica, SQL Server Integration Services(SSIS), Azure Data Factory, DBT, Pentaho Data Integration, Matillion, Apache Nifi

Data Analysis Tools

NumPy, Pandas, Power BI, Power Pivot, Power View, Power Map, Apache Spark, Tableau

Data Compression

PostgreSQL Data Compression Strategies, Hadoop Compression Techniques

Backup & Recovery

Hadoop Cluster Backup and Recovery, Snowflake Disaster Recovery Plans, AWS Lambda Auto-scaling Configurations

Data Visualization Integration

Power BI Dashboards, Tableau and Power BI Integration, Embedding Tableau Visualizations

PROFESSIONAL EXPERIENCE:

Client: BCBS, Jacksonville, FL Oct 2020 – Present

Roles: Sr. Data Engineer

Responsibilities:

Proficient in creating joins for queries involving multiple tables.

Proficient in Data Warehouse concepts, OLTP, OLAP, Star Schema, Snow flake Schema, Fact and Dimension tables.

Proficient with T - SQL to create Tables, Views and Stored Procedures.

Proficient in Normalization and De-Normalization of existing tables for faster query retrieval.

Strong knowledge inPowerBIon how to import, shape, and transform data for business intelligence (BI), Visualize data, author reports, schedule automated refresh of reports and create and share dashboards.

Extensively used custom R-Visuals, Clustered Bar Charts, Clustered Column charts, Donut Charts, Hierarchy slicers, Waterfall charts, Scatter charts, Area charts, KPI s in power BI reports.

Extensively involved in creating robust complex power BI models, writing complex DAX logic s involving rank, filters and data massaging.

Maintained Cognos report documentation for easy reference and knowledge sharing within the team. Resolved user issues and provided timely support for Cognos report-related queries or concerns.

Regularly reviewed and updated Cognos reports to reflect changing business needs and data sources.

Contributed to the evolution of reporting capabilities by exploring new features and technologies in Cognos.

Developed and maintained Cognos reports to ensure accurate and timely data visualization.

Implemented data governance practices to uphold data integrity and compliance standards in Cognos reports.

Collaborated with business stakeholders to gather requirements and ensure alignment of Cognos reports with organizational goals.

Optimized report performance by fine-tuning queries and enhancing data models in Cognos.

I worked directly for the IT director over the division of business analytics at Confidential. I managed all Power BI development within this position. I was response for maintaining the views as well as the stored procedures used within the Power BI structure. I also handle all administration processes dealing with the Power BI Gateway.

Managed all refreshes for the Power BI reports within our environment

Managed/performed all development for Power BI. Installed the Power BI gateway onto the server

Managed and created all datasets on that Power BI gateway

Managed all scheduling of refreshing of the Power Bi reports

Maintained and wrote the views for the Power BI schema

Extensively used visual filters, page filters, report filters and drill through filters in power BI reports.

Designed, developed and deployed reports, sub-reports, drill down reports, and drill through reports and parameterized reports and adding new features to the reports in MS SQL Server environment using SSRS.

Possess technical skills in defining Source and target metadata definitions for Source to Target mapping specification.

Designed and developed SSIS packages to extract data from the transactional and/or operational databases, transformed the data according to analytical / reporting business needs, and loaded the data into a data warehouse or data mart.

Worked with subject matter experts (SME's) and project team to identify, define, collate, document and communicate the data migration requirements.

Environment: SQL (T-SQL), Data Warehouse concepts, OLTP, OLAP, Star Schema, Snowflake Schema, Fact tables, Dimension tables, Normalization, De-normalization, Power BI, R-Visuals, Cognos, DAX, Cognos, Data governance, Data integrity, Compliance standards, SSRS, SSIS.

Client: SunTrust Bank, Atlanta, GA Apr 2016 – Sep 2020

Role: Power BI Data Engineer

Responsibilities:

Implemented information into CSV records and stored them in AWS S3 by using AWS EC2 and loading them into AWS Redshift.

Worked on code migration of quality monitoring tool from AWS EC2 to AWS Lambda and built logical datasets to administer quality monitoring on snowflake warehouses.

Used AWS Athena to transform and clean the data before it was loaded into data warehouses.

Used Informatica Power Center Designer to analyze the source data to Extract & Transform from various source systems by incorporating business rules using different objects and functions that the tool supports.

Developed comprehensive error handling mechanisms within Informatica workflows, leveraging logging features to facilitate quick identification and resolution of data integration issues.

Used Python scripts to update the content in the database and manipulate files.

Authored Python Scripts for custom UDF’s for Row/ Column manipulations, merges, aggregations, stacking, data labeling, and for all Cleaning and conforming tasks.

Involved in writing T-SQL working on SSIS, SSAS, Data Cleansing, Data Scrubbing, and Data Migration.

Designed geospatial data in PostgreSQL databases, supporting location-based queries and applications.

Implemented data sharding techniques to distribute large datasets across multiple nodes, enhancing NoSQL database scalability.

Designed and implemented effective indexing strategies to accelerate query execution and enhance the retrieval speed of data from MongoDB collections.

Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs and Scala.

Automated the data processing with Oozie to automate data loading into the Hadoop Distributed File System.

Designed and implemented data warehousing solutions using Scala, facilitating efficient storage and retrieval of structured data.

Integrated Hadoop seamlessly with other big data tools and technologies, such as Apache Spark, Hive and Pig, to create end-to-end data processing pipelines.

Developed Spark applications using Scala and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats.

Developed deployment scripts and integrated Kafka components into continuous integration/continuous deployment (CI/CD) pipelines, streamlining development workflows.

Implemented robust disaster recovery plans for Kafka, ensuring minimal data loss and rapid system recovery in the event of unexpected outages or failures.

Developed strategies for scaling Snowflake resources based on data growth projections, ensuring platform's ability to handle increasing data volumes effectively.

Developed and tested disaster recovery plans within Snowflake, ensuring data resilience and minimizing potential data loss or downtime in case of unexpected events.

Employed custom scripting and calculated fields in Tableau for advanced data manipulations, allowing for intricate analysis and tailored visualizations.

Developed and maintained Cognos reports to ensure accurate and timely data visualization.

Managed the entire data lifecycle within Tableau, from data acquisition and integration to archiving and purging, ensuring a well-organized and efficient data environment.

Utilized Power BI's data cleansing capabilities to clean and preprocess raw data, ensuring data quality and accuracy for reporting and analysis.

Skilled in managing code repositories on GitHub, implementing best practices for code organization, and ensuring compliance with industry standards.

Implemented Docker-based solutions for data storage and retrieval, including database containers and data lakes, to ensure data availability and accessibility.

Environment: AWS (EC2, S3, Redshift, Lambda, Athena), Snowflake, Informatica, Python, T-SQL, SSIS, SSAS, PostgreSQL, NoSQL, MongoDB, Hive, Spark RDDs, Scala, Oozie, Hadoop, HDFS, Kafka, Tableau, Microsoft Power BI, GitHub.

Client: CITRIX, Bangalore, India. July 2014 – Jan 2016

Role: Data Engineer

Responsibilities:

Developed and implemented data quality checks and validation processes within Azure Data Lake and Azure Data bricks to ensure high-quality and accurate data for analytics and reporting.

Designed scalable architectures, utilizing Azure Blob Storage's scalability features to accommodate growing data volumes and evolving business requirements.

Implemented effective data partitioning and indexing strategies to improve query response times in Azure Synapse Analytics, optimizing data storage and retrieval.

Implemented security measures within Informatica environments, ensuring data confidentiality, integrity, and availability, while adhering to industry best practices and compliance standards.

Implemented data quality checks and validation processes within Python scripts to identify and address discrepancies or anomalies in the data.

Implemented intermediate functionalities like events or records count from the flume sinks or Kafka topics by writing Spark programs in Python.

Involved in PL/SQL query optimization to reduce the overall run time of stored procedures.

Utilized and configured monitoring tools specific to NoSQL databases, providing real-time insights into performance metrics and system health.

Implemented data compression strategies in PostgreSQL databases to optimize storage efficiency and reduce disk space utilization.

Skilled in utilizing Hadoop's map Reduce and Spark frameworks to process and transform large volumes of data, optimizing performance and resource utilization.

Implemented data security measures in Scala, including encryption and access controls, to safeguard sensitive information throughout the data engineering workflow.

Implemented robust backup and recovery strategies for Hadoop clusters, ensuring data integrity and minimizing the risk of data loss in case of failures.

Conducted regular performance audits and optimizations, fine-tuning Kafka configurations and addressing bottlenecks to achieve optimal throughput and latency.

Developed multiple POC’s using Spark, and Scala and deployed them on the Yarn Cluster, compared the performance of Spark, with Hive and SQL.

Created automated data pipelines in Snowflake, streamlining the movement and transformation of data, resulting in increased operational efficiency.

Enforced Tableau best practices for dashboard design, data preparation, and visualization development to maintain a high standard of quality.

Integrated Tableau with other data platforms and tools, fostering a cohesive data ecosystem and enabling seamless data flow across the organization.

Used Power BI and Power Pivot to develop data analysis prototype, and used Power View and Power Map to visualize reports.

Published Power BI Reports in the required originations and created Power BI Dashboards available in Web clients and mobile apps.

Environment: Azure (Data Lake, Data bricks, Blob Storage, Synapse Analytics), Informatica, Python, Apache Flume, Apache Kafka, Spark, PySpark, PL/SQL, NoSQL, PostgreSQL, Hadoop, Map Reduce, Scala, Hadoop, Kafka, Snowflake, Tableau, Power BI, Power Pivot.

Client: Kelly Max Technologies, Hyderabad, IN Apr 2012 – Jun 2014

Role: Data Analyst

Responsibilities:

Utilized Python, Pandas, NumPy, and SQL to clean, transform, and analyze large datasets from various sources such as AWS S3, MySQL, and PostgreSQL.

Employed AWS Glue and AWS Data Pipeline for efficient ETL processes, ensuring data accuracy and integrity.

Created interactive dashboards and reports using Tableau, Power BI, and Excel to visualize key performance indicators (KPIs) and trends, enabling stakeholders to make data-driven decisions.

Conducted statistical analysis using SAS and SPSS to identify patterns, correlations, and trends in datasets, facilitating informed business decisions.

Designed various relational databases including MS-SQL, Sybase, MySQL, Oracle, PostgreSQL, and NoSQL databases like MongoDB for data storage and retrieval.

Implemented machine learning algorithms and statistical models in Alteryx and Jupyter Notebooks to perform predictive analysis and anomaly detection on large datasets.

Designed and maintained data pipelines using Hadoop ecosystem tools such as Apache Spark and Apache Hadoop, ensuring efficient data processing and storage.

Conducted A/B testing experiments to optimize website content and design based on data-driven insights gathered from Google Analytics.

Developed PowerShell scripts to automate routine system administration tasks. Provided training and support to team members on JIRA usage, including best practices for project management, task tracking, and collaboration. Implemented Snowflake security policies and access controls to enforce data privacy and compliance with regulatory requirements, such as GDPR and CCPA.

Created templates and style guides in Microsoft Word for consistent and professional-looking documents across teams and projects.

Managed relational databases such as MySQL, Oracle, and PostgreSQL, optimizing query performance and ensuring data availability.

Applied NLP techniques using NLTK and SpaCy to extract insights from unstructured text data, such as customer feedback and social media posts.

Designed and implemented ETL processes using AWS Glue, Alteryx, and custom scripts to extract data from diverse sources, transform it into usable formats, and load it into target systems.

Environment: Python, Pandas, NumPy, SQL, AWS Glue, AWS Data Pipeline, Tableau, Power BI, Excel, SAS, SPSS,

MS-SQL, Sybase, MySQL, Oracle, PostgreSQL, MongoDB, Jupyter Notebooks, Apache Spark, Apache Hadoop, Google Analytics, PowerShell, JIRA, Snowflake, Microsoft Word, SpaCy.

Client: GoodRx, Santa Monica, CA. Jun 2009 - Feb 2010

Role: SQL Developer

Responsibilities:

Created indexes on selective columns to speed up queries and analysis in SQL Server Management Studio.

Responsible for design and development of modules in Microsoft Access using VBA coding for internal applications

Developed complex Stored Procedures and views to generate various Drill-through reports, parameterized reports and linked reports using SSRS.

Deployed SQL server 2016 database to Azure database by scripting the database, all the objects and scripting out insert data script for master tables.

Checked in all SQL objects schema, deployment scripts and related queries in the TFS (Team Foundation Server).

Interfaced with various teams within the company and considered a Subject Matter Expert on databases.

Created SSIS Packages using Pivot Transformation, Fuzzy Lookup, Derived Columns, Condition Split, Term extraction, Aggregate, Execute SQL Task, Data Flow Task, and Execute Package Task etc to generate underlying data for the reports and to export cleaned data from Excel Spreadsheets, Text file, MS Access and CSV files to data warehouse.

Performed data transfers using BCP and BULK INSERT utilities. Was involved in the design and implementation of Disaster Recovery.

Environment: SQL Server Management Studio, Microsoft Access, SSRS (SQL Server Reporting Services), SQL Server 2016, TFS, SSIS, Pivot, Fuzzy Lookup, Condition Split, Execute SQL Task, Data Flow Task, Execute Package Task, BCP, BULK INSERT, Disaster Recovery.

Client: Smart IMS, India. Aug 2008- April 2009

Role: SQL Developer

Responsibilities:

Developed ETL packages using SSIS/Informatica Power Designer for various customers to fetch data from different EDW sources.

Loading (ETL) data into the Data warehouse from Heterogeneous databases such as Oracle, DB2, Sysbase and SQL server.

Worked on different LOBS related to insurance which are Personal Lines (Homeowners, Dwelling, FAL and Farm Owners) and Commercial Lines (General Liability, Crime, Inland Marine, Commercial Auto Liability/No Fault/Physical damage and Fire and Allied Lines).

Worked on UNIX shell scripting in automation of ETL Jobs.

Creation of logs from the UNIX shell scripts for auditing the automated Workflow process.

Working on Agile Methodologies with daily scrum meetings and share point/Microsoft Planner to update and monitor tasks.

Processing the Interface Data using the SQL Loader by loading the data using Flat Files.

Working on Adhoc Reports by writing Complex SQL’s using TOAD and SQL Developer.

Enhanced the existing ECA (Error Correction Application) application based on the client requirements.

Environment: SQL server Management Studio for different client Database, Visual Studio Management Studio, Cisco Any Connect with client Database, JIRA.

EDUCATION: JAWAHARLAL NEHRU TECHNOLOGOICAL UNIVERSITY Hyderabad, TS, India

BTech in COMPUTER SCIENCE AND ENGINEERING June 2002 - May 2006

Major in Computer Science.

Contact this candidate