Blogs

This exam evaluates architectural and design skills with understanding of the data ecosystem. Snowflake experience is essential to start preparing for this certtification.

This guide is summary of key areas required to be studied for certification exam. The first part of blog shares details on exam , pattern, pre-requisites before you start preparing for certification. 

I am excited to learn about Snowflake Query_insights view and how this can be of a help to improve performance of queries. Snowflake’s focus on simplicity and ease of use always excites me most about the platform, features, and services.

It was snowing in SFO in the month of June! I attended Snowflake summit this year and had an amazing experience listening to keynotes, sessions with lots of takeaways. It’s tough to cover everything in one blog! This is summary blog of the annoucements. Look out for upcoming blogs to get deeper insights of each of the managed services, services, and features announced this year. 

All about understanding the Snowflake AI Assistant — Snowflake Co-Pilot and using it to develop, optimize, get recommendations, generate sql queries and more. Lets begin with series of questions to understand more!

This is the next blog in the series about understanding and using Snowflake Cortex functions. The first blog helps you to understand the remaining Snowflake Cortex functions and sample use cases to implement them.

This blog is the next blog in the series of “Data Quality” and “Data Governance” series. Refer to my earlier blog on understanding data metrics functions.

This blog is the next blog in the series of “Data Quality” and “Data Governance” series. Refer to my earlier blog on “Data Quality Essentials” that talks about necessity of Data Quality and metrics considered to validate data quality frameworks.

This is the next blog in the series about understanding and using Snowflake Cortex functions. The first blog helps you to understand the remaining Snowflake Cortex functions and sample use cases to implement them.

This blog in about understanding and using Snowflake Cortex functions. The first blog helps you to understand the Snowflake Cortex and its functions offered as part of the Cortex.

Snowflake Cortex is a managed service that offers AI and ML solutions to users. There are two types of functions offered by Snowflake:

1. LLM Functions: These are SQL and Python-based functions that can be used to develop an understanding, query, translate, summarize, and generate free-form text.

2. ML-Based Functions: These are SQL functions based on the ML algorithms that are used to derive predictions.

Data integrations are integral part of the data platform implementations. There are various integrators and tools available that can be used to integrate the applications or systems with Data platform. In a typical data platform architecture, you would have the source integrations that are required to be integrated to bring in the data to as source to the platform. You will also have bunch of consumer integrations and applications that runs on top of your “data” platform or consume “data” from your platform. In this blog, you will learn about the import and export of data required as part of source and consumer application integrations.

The data use cases blog series focuses on data analytics, consumption of the data, and data accessibility across regions, platforms, and users. This blog series starts with Snowflake offerings for data consumers. 

This blog focuses more on implementing the dashboards and monitoring techniques that will help to optimize the performance and cost as well as help in implementing the controls to monitor and report the utilization.

This blog focuses more on implementing the optimization techniques which will help to optimize the performance and cost as well as help in implementing the controls to monitor and report the utilization.

Learn more on the optimization services available in Snowflake.

Snowflake’s compute or query processing layer is one of the layers of its architecture. Compute is referred to as Warehouse. Warehouses are available in various sizes and their consumption is calculated based on the usage in the form of credits. The earlier blog refers to the credit-to-cost computation. This blog refers to computing the warehouse cost and components that are considered as part of compute cost.

This blog focuses more on understanding the various components of Snowflake that contribute to the total cost. You will learn more about the cost in terms of Q&A.

Snowflake’s costing model is very transparent and easy to understand. You will be charged only for the services being used and only for the time they are up and running. You need to understand Snowflake’s architecture — three layered architecture and their features to understand the costing model well. You can refer to my earlier blogs to understand some of the foundational concepts, features required for the cost series.

This blog focuses more on query tags , defining query tags, using query tags , use cases and benefits of using query tags.

Data security is one of the critical pillar of Data governance architecture. Column level and Row level security is part of data security implementation. Earlier blogs in Data Governance series focus on column level security. This blog focuses on implementing Row level security in Snowflake.

This is the third blog in the Data Governance series focusing more on implementing tag-based policies in Snowflake.

This blog helps you understand tagging in Snowflake. This also covers tagging details — how you can create them, use them, assign to the database objects, and track them for usage.

This blog is designed to share information on Snowflake Event tables. This blog covers details of Event tables — creation, usage, use cases to implement them, querying event tables, etc.

This is the next blog in the series that helps you learn more about data lake, typical data lake challenges, designing a data lake and implementing it with Snowflake.

Data governance implementation has various pillars. The data governance umbrella consist of many subcategories. Some of the pillars include — Access control, Data protection (classification), Data security (encryption ) , Data Quality, Data sharing etc. This blog will help you learn one of the most important security feature — Data Protection.

You must have come across many scenarios, use cases or business requirements where you need to generate data , share or send over to other applications for their consumption.

In a typical implementations, you might have developed and deployed set of jobs, pipelines to generate aggregates or consumer data to be shared either by FTP or ETL jobs etc.

Snowsight is Snowflake’s web user interface (UI). Snowflake also had Classic console — earlier Web UI which is available to old accounts as an option in Snowsight to access previous version of Web UI. With the recent release, Snowflake announced that Snowsight will be the default user interface for all accounts that are crated post May 2023.

Data Mesh is an architectural approach and organizational model for data management that aims to address the challenges of scalability, autonomy, and agility in large-scale data systems. It proposes a paradigm shift from centralized data platforms to a decentralized and domain-oriented approach.

A data warehouse is a centralized repository of integrated data that is used for reporting, data analysis, and business intelligence purposes. It is designed to support the decision-making process by providing a single, comprehensive view of a business’s data from various sources.

You are designing a centralized data platform and keen on leveraging the platform capabilities to implement data integrations, develop data pipelines, and implement data analytics or AI/ML.

Alerting or Alerts play a crucial role in any application, system implementation. Alerts are notifications, messages setup to notify existing issues as well as proactive notification upon reaching up given threshold values, limits of utilizations. We can refer these alerts as part of reactive and pro-active alerting mechanisms.

We are going to learn more about implementing slowly changing dimensions or Change data capture in Snowflake using Snowflake — Streams.

In this blog, we are going to learn more about data protection features. Snowflake has two different features — Time Travel and Failsafe.

In this blog, we are going to learn more about data protection features. Snowflake has 2 different features — Time Travel and Failsafe.

We are going to learn more about account management and how we can automate account management process using Snowsight dashboards.

we are going to learn more about Snowsight — new Web UI. Snowsight is used to build dashboards and used natively with Snowflake.

In this blog, we are going to learn about Data Cloning — one of distinguishing feature of Snowflake.

Snowflake feature — Data Sharing is feature which allows to create/share objects required to be accessed or utilized by other teams. Data sharing enables users to share snowflake data objects like tables, external tables, secure views, secure UDFs, materialized views etc. 

Snowflake is data on cloud with shared data architecture. There are many features of Snowflake while some of them are key features which makes Snowflake different than any other Datawarehouse offerings on any other cloud.

Snowflake offers Data on Cloud offered as SaaS — Software as a Service . Snowflake is not build on any other Database or Hadoop/Bigdata platform. This is data offering built from scratch for cloud offering.

GCP BigQuery is a serverless and cost-efficient data service of GCP. You can use GCP BQ to implement a data platform — data warehouse, or data lake on GCP. GCP BQ also offers ML implementation to integrate ML use cases in an SQLish way. BQ ML allows users to create and maintain models using BQ SQL. There are pre-trained models available to be used in BQ ML.

Cloud Data Fusion is a fully managed, cloud-native, enterprise data integration service for quickly building and managing data pipelines. Business users, developers, and data scientists can easily and reliably build scalable data integration solutions to cleanse, prepare, blend, transfer, and transform data without having to wrestle with infrastructure.

ELT — Extract, Load and Transform — Building ELT pipelines are little different than building an ETL pipeline. There is difference in approach of implementation with these 2 types — ETL & ELT.

Google Cloud’s serverless platform help us to build, develop, and deploy functions and applications, as source code or containers, while simplifying the developer experience by eliminating all infrastructure management.

In this blog, we are going to see some of the important aspects of designing/architecting ELT on GCP specifically when we have chosen Bigquery as DW/Data Lake to store data. Before explaining the specifics, lets look at the Bigquery and try to find out answer to many questions like access to data, retrieve data, data security, data loss prevention, run data analysis , data analytics, generate stats , building dashboards, exploring data through reporting and the most importantly how can we get ML models built/run on Bigquery? 

We have seen different types of change data capture and ways to implement change data capture. Slowly Changing Dimensions(here after referred as SCD) can be implemented in many ways out of which SCD Type II is widely used enterprise solution. Let’s quickly recap on SCD Type II — This is Row versioning , Track changes as version records with current flag & active dates and other metadata. Type II implementations carry few of audit columns with each of its implementation to represent audit trail and active records.

Datastream is a serverless and easy-to-use change data capture (CDC) and replication service. It allows you to synchronize data across heterogeneous databases and applications reliably, and with minimal latency and downtime.

This blog shares some of design recommendations based on my previous experiences of GCP Data migrations , DW implementation with Google Bigquery.

Data security is securing data wherever its located, whether its in transit or stored or at rest, whether its managed/maintained by internal or external teams. This is critical to implement and ensure data security is in place whether data is in cloud or at on-premises systems.

This blog shares recommendations to optimize BigQiery performance.

This blog shares BigQuery pricing models and optimization techniques for BQ pricing.

Migrations are always challenging. I have worked on various migrations — Legacy to Legacy, Legacy to Bigdata and now from past few years, working on migrations to Cloud platform. Based on my past experiences, I am coming up with series of blogs on Migrations. Hope this blog series will help to plan, design, and implement the processes to migrate your applications/workloads to Google cloud.

This blog is second chapter of migration series, we are going to discuss more about migration path and phases of migration. In earlier blog , we have learnt that the assessment of existing system , defining goal, defining type of migration applicable to current project/application.

This blog is third and last but one chapter of migration series, we are going to discuss some of the reference architectures and use cases of GCP Migration. In earlier blogs, we have learnt that the assessment of existing system , defining goal, defining type of migration applicable to current project/application. We also learnt the phases and designing phases of GCP Migrations.