Blog

<h2>Geospatial Analysis in Sigma Computing – Working with location-based data.</h2>Geospatial data is everywhere—from mapping a delivery route to tracking weather patterns. With Sigma Computing, we can take advantage of geospatial data to uncover location-based insights quickly and easily.<h3>What Is Geospatial Data?</h3>Sigma Computing supports a special data type known as geography. This data type is designed to store structured spatial data that represents geographic locations or features. Here’s what we need to know:<ul><li>Structured Spatial Data:<ul><li>Contains information like points, lines, and areas that represent real-world locations.</li><li>Can be used to map customer locations, distribution centers, or even regions of influence.</li></ul></li><li>Primary Uses:<ul><li>Creating maps for visualization.</li><li>Performing spatial joins to compare and contrast different geographic features.</li></ul></li><li>Supported Data Sources:<ul><li>Sigma directly supports the geography data type when connected to Snowflake and BigQuery.</li><li>When using other connections (like different cloud data warehouses or DBMS), Sigma retrieves data in standard geospatial formats such as Well-Known Text (WKT), though it presents them as plain text.</li></ul></li></ul><h3>Understanding Geography Data Formats</h3>Geography data can be represented in multiple formats, each of which visually or textually describes different kinds of geographic features. Here are the key formats and what they represent:<ul><li>Well-Known Text (WKT): A textual markup language for representing vector geometry objects.</li></ul>Examples:<ul><li style="list-style-type: none;" data-mce-style="list-style-type: none;"><ul><li>POINT: POINT (10 20)</li></ul></li></ul>Represents a single location defined by X (longitude) and Y (latitude) coordinates.<ul><li style="list-style-type: none;" data-mce-style="list-style-type: none;"><ul><li>LINESTRING: LINESTRING (30 40, 50 60, 70 80)</li></ul></li></ul>Describes a sequence of points connected by straight lines, useful for mapping roads or paths.<ul><li style="list-style-type: none;" data-mce-style="list-style-type: none;"><ul><li>POLYGON: POLYGON ((0 0, 10 0, 10 10, 0 10, 0 0))</li></ul></li></ul>Defines an area with a boundary, such as a city or a country.<ul><li style="list-style-type: none;" data-mce-style="list-style-type: none;"><ul><li>MULTIPOINT, MULTILINESTRING, and MULTIPOLYGON:</li></ul></li></ul>These formats combine multiple points, lines, or polygons into a single entity. They’re useful when we need to represent several related features together.<ul><li>Why Use These Formats?<ul><li>They standardize how geographic information is stored and shared.</li><li>They allow for a wide range of spatial operations (like distance calculations, overlaps, or containment tests).</li></ul></li></ul><h3>Building Geography Map Charts in Sigma</h3>Creating a visual map that shows our geospatial data can add significant value to our analysis. Let’s walk through how we can build a geography map in Sigma.SIgma Computing offers 3 types of Map charts-<ul><li>Map - Points<ul><li>Point maps require a numerical column on both the map's LATITUDE and LONGITUDE fields. </li></ul></li></ul><figure id="figure-xi5ub14s7m820qdq9"><img id="xi5ub14s7m820qdq9" src="https://cdn.durable.co/blocks/2eQHjyOR9xhjb38GgpY20Hn7so8uuKPPeBJgUSX9cqyifVXrREUTPX294CuESeeL.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/2eQHjyOR9xhjb38GgpY20Hn7so8uuKPPeBJgUSX9cqyifVXrREUTPX294CuESeeL.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><ul><li>Map - Region<ul><li>Region maps require a single text Columns in the Region property. Column values must match one of the following region types and be consistent within the column.</li></ul></li></ul><figure id="figure-id7oxgobbsm821p7dd"><img id="id7oxgobbsm821p7dd" src="https://cdn.durable.co/blocks/2eJasGD10CiAtsriyZGViDVj9HfFin509wD6DgMkonWFzijTAT4ely5z4aZXLvhL.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/2eJasGD10CiAtsriyZGViDVj9HfFin509wD6DgMkonWFzijTAT4ely5z4aZXLvhL.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-6kufmk9hgm821qsyt"><img id="6kufmk9hgm821qsyt" src="https://cdn.durable.co/blocks/3ye5uEOYK2L4NXsC9euc7UD0u52d47sRHI0LIDJR5KaU1eqVlcXISP2zedu7xjpl.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/3ye5uEOYK2L4NXsC9euc7UD0u52d47sRHI0LIDJR5KaU1eqVlcXISP2zedu7xjpl.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><ul><li>Map - Geography<ul><li>WKT and JSON formats allow the storage of more complex geographical data than simple numeric lat/long columns.</li><li>Sigma's Map - Geography chart requires a geography or variant data column on the map's Geography field. If our WKT or GeoJSON data is stored as a text column (abc), we can convert it using the Geography or Variant function.</li><li>GeoJSON data must also have "coordinates" nested within a "geometry" key in order to be plotted. In GeoJSON and our geography type, the coordinate order is always longitude, latitude.</li></ul></li></ul>ExamplesFor example, given the following GeoJSON data:<figure id="figure-c47ke9si7fm8223mxi"><img id="c47ke9si7fm8223mxi" src="https://cdn.durable.co/blocks/4WcJdO2n9WWdMJS78vkSuEvyzTHeCprF88Kzi1kM1dtZMcXJrbW5J4Y6hIOx3bb7.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/4WcJdO2n9WWdMJS78vkSuEvyzTHeCprF88Kzi1kM1dtZMcXJrbW5J4Y6hIOx3bb7.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>Below is a non-working example, even though it is valid GeoJSON, as it lacks the "geometry" key:<figure id="figure-twpcyi8kgsm82266q6"><img id="twpcyi8kgsm82266q6" src="https://cdn.durable.co/blocks/3aWBqr2f84Mw0X08TFN8yIt8IazOJiVA3FWXFbAyPoJvUgWNgRyZZs6RkZ2tRqpV.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/3aWBqr2f84Mw0X08TFN8yIt8IazOJiVA3FWXFbAyPoJvUgWNgRyZZs6RkZ2tRqpV.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><h3>Steps to Build Your Map</h3><ul><li>Step 1: Connect to Your Data Source<ul><li>Confirm the data source (e.g., Snowflake or BigQuery) supports the geography data type.</li><li>For other connections, geospatial data might appear as text—use Sigma’s Geography or Variant functions to convert it.</li></ul></li><li>Step 2: Prepare Your Data<ul><li>Clean the data and ensure that geography columns are in the proper format (WKT or correctly structured GeoJSON).</li><li>Validate and convert text-based geospatial data using Sigma’s built-in functions if needed.</li></ul></li><li>Step 3: Create a New Worksheet in Sigma<ul><li>Open a new worksheet dedicated to your map.</li><li>Drag and drop the converted geography field into the visualization area.</li></ul></li><li>Step 4: Configure the Map Visualization<ul><li>Choose the map type that suits your data:<ul><li>Region Map: For text-based identifiers like state or country names.</li><li>Point Map: For numeric latitude and longitude columns.</li><li>Geography Map: For complex WKT or GeoJSON data.</li></ul></li></ul></li><li>Adjust layers, colors, and labels to enhance clarity.</li></ul><h3>Geospatial Joins in Sigma: Intersect and Within</h3>One of the standout features in Sigma Computing is the ability to perform geospatial joins. Geospatial joins helps us answer questions like “Which customer locations lie within a sales region?” or “Which roads cross a specific area?” Sigma offers two main operators for these tasks:<ul><li>Intersect Operator:<ul><li>What It Does:<ul><li>Returns rows from both datasets when their geospatial objects overlap or intersect.</li></ul></li><li>How to Think About It: <ul><li>Imagine two circles on a map. If they overlap, the intersect operator finds those overlapping areas.</li></ul></li><li>When to Use It:<ul><li>Finding common areas between two spatial datasets.</li><li>Mapping where two different features (like service zones and delivery areas) touch.</li></ul></li></ul></li><li>Within Operator:<ul><li>What It Does: <ul><li>Returns rows where geospatial objects from one dataset are entirely contained within those of another.</li></ul></li><li>How to Think About It:<ul><li>Picture a small park inside a larger city boundary. The “within” operator would show that the park is entirely within the city.</li></ul></li><li>When to Use It:<ul><li>Identifying if a point (like a store location) falls completely inside a predefined boundary (such as a service area).</li></ul></li></ul></li></ul>Practical Example:Imagining we have two datasets:<ul><li>Dataset A: Contains customer addresses as points.</li><li>Dataset B: Contains sales territories as polygons.</li></ul>Using the within operator, we can easily find which customers are served by which sales territory. Alternatively , if we want to see areas where customer clusters and sales territories overlap, the intersect operator comes into play.<figure id="figure-5dchc7w7ym822a2av"><img id="5dchc7w7ym822a2av" src="https://cdn.durable.co/blocks/3nhVBDw5SsbRhFQpQDazNLFMKiQPx9yJFEpHXQglnO8XMEKC0SZOLme6jOMozTOA.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: undefined% 50%;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined%" data-positiony="50%" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/3nhVBDw5SsbRhFQpQDazNLFMKiQPx9yJFEpHXQglnO8XMEKC0SZOLme6jOMozTOA.png" data-mce-style="aspect-ratio: 16/9; object-position: undefined% 50%;"></figure>Explanation:<ul><li>Customer Data &amp; Sales Territory Data : Represent the two datasets.</li><li>ST_WITHIN (or) INTERSECT: The join operation where the spatial function (either ST_WITHIN or INTERSECT) is applied.</li><li>Output: The final output, which lists customers along with their corresponding sales territory.</li></ul><h3>Conclusion</h3>Working with geospatial data in Sigma Computing opens up a new dimension of analysis by allowing us to incorporate real-world location data into our decision-making process.Whether we’re mapping retail locations, planning distribution routes, or analyzing regional trends, Sigma’s geospatial capabilities make the process efficient and accessible—even for users with limited technical expertise.

Working with Geospatial Data in Sigma Computing

<h2>Transforming Data with Pivot Tables in Sigma Computing</h2>When it comes to analyzing large datasets, pivot tables are a game-changer. They allow us to extract meaningful insights quickly, without needing complex calculations. Sigma Computing has taken pivot tables to the next level, enabling teams to work with data in real-time, making it easier to explore trends and make informed decisions.In this blog, we’ll dive deep into what pivot tables are, how they function within Sigma Computing, and explore the latest features (including beta-testing updates) to enhance our data analysis skills.<h3>Understanding Pivot Tables</h3>A pivot table is a powerful data summarization tool that helps us re-arrange and aggregate data for a clearer view of patterns and trends. Instead of looking at a massive dataset, we can view a summarized version that highlights trends and patterns in a compact format.Key Concepts:<ul><li>Data Summarization : <ul><li>Condenses large datasets into manageable summaries.</li><li>Allows us to focus on key metrics rather than individual data points.</li></ul></li><li>Grouping and Categorization :<ul><li>Organizes data into logical groups (for example, by region, product type, or time period).</li><li>Helps in comparing different segments side by side.</li></ul></li><li>Aggregation :<ul><li>Calculates totals, averages, counts, and other summary statistics.</li><li>Offers insight into overall performance as well as details within each group.</li></ul></li><li>Filtering and Sorting :<ul><li>Enables us to focus on specific subsets of data.</li><li>Sorting helps arrange data in ascending or descending order for easier comparison.</li></ul></li></ul>In Sigma Computing, these pivot tables update in real-time, ensuring that every team member sees the most current information at all times.<h3>How Pivot Tables Work in Sigma Computing</h3>Sigma Computing makes creating a pivot table simple and flexible. Here's how we can start working with pivot tables in Sigma:<ul><li>Creating a Pivot Table :<ul><li>Select the data you want to analyze.</li><li>From the toolbar, choose Pivot Table from the options.</li><li>Sigma will generate a default pivot table based on your data. </li></ul></li></ul><figure id="figure-v83v06ltrem7srskab"><img id="v83v06ltrem7srskab" src="https://cdn.durable.co/blocks/20yEDB3uoUKxKsFL66CeSXV3x5O9DBmfUAOveMWrGcrsq07isw6NN2TNXCu6nP0M.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/20yEDB3uoUKxKsFL66CeSXV3x5O9DBmfUAOveMWrGcrsq07isw6NN2TNXCu6nP0M.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><ul><li>Configuring Rows, Columns, and Values:<ul><li>Drag fields to the Rows and Columns to structure the data.</li><li>The Values section is where you specify how to aggregate the data (e.g., Sum, Average, Count).</li><li>For example, you might have “Region” as a row, “Product” as a column, and sales “Amount” as the value.</li></ul></li></ul><figure id="figure-6atozaidhim7su0p3i" class=""><img id="89h9j47v7fm7srzeos" src="https://cdn.durable.co/blocks/36U31Hp7hF3dsgqk5LFtHGQ3l3j90haiRiKHdgcCaUjPCHiWp0Dby4DLfA4S4D8Q.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/36U31Hp7hF3dsgqk5LFtHGQ3l3j90haiRiKHdgcCaUjPCHiWp0Dby4DLfA4S4D8Q.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"> </figure><ul><li>Adding Filters and Sorting:</li><li style="list-style-type: none;" data-mce-style="list-style-type: none;"><ul><li>Filters allow you to focus on specific subsets of data (e.g., showing only sales data from the last quarter).</li><li>Sorting options let you arrange your data in ascending or descending order for easier analysis.</li></ul></li></ul><figure id="figure-buw2jox3hlm7su211k"><img id="buw2jox3hlm7su211k" src="https://cdn.durable.co/blocks/3a8a1cnROB0IUGNzU8VsZcvS1oBJPqqj3guiWjeZUNcOKF1jxbxCSgcdfIse0XmQ.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="null" data-positiony="null" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/3a8a1cnROB0IUGNzU8VsZcvS1oBJPqqj3guiWjeZUNcOKF1jxbxCSgcdfIse0XmQ.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><h3>Customizing our Pivot Table</h3>Once we’ve set up the basic structure, customizing our pivot table can significantly enhance its readability and usefulness. Here are some personalization options we rely on:<ul><li>Conditional Formatting :<ul><li>Automatically highlight important values using color coding.</li><li>For example, we can set cells to appear in green when sales exceed a certain threshold or red when they fall below target levels.</li></ul></li></ul><figure id="figure-7cdnjl86vpm7stzfqw"><img id="7cdnjl86vpm7stzfqw" src="https://cdn.durable.co/blocks/2dAp3ZncTx6XmAOcYTrEqKsJNh7HJ7XjTYIP7IxcnKoNgSDPmt1mk4JvpHfmcd5T.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: null% 50%;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="null%" data-positiony="50%" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/2dAp3ZncTx6XmAOcYTrEqKsJNh7HJ7XjTYIP7IxcnKoNgSDPmt1mk4JvpHfmcd5T.png" data-mce-style="aspect-ratio: 16/9; object-position: null% 50%;"></figure><ul><li>Reordering Fields :<ul><li>Experiment with dragging and dropping different fields to see the same data from various perspectives.</li><li>This flexibility can reveal hidden trends or highlight key correlations that might not be obvious at first glance.</li></ul></li></ul>Customizations like these allow us to modify the pivot table so that it not only looks good but also communicates the data clearly and effectively.<h3>Advanced Functionalities</h3>Sigma Computing continuously evolves its pivot table features, making it easier for us to extract even deeper insights. Here are some advanced functionalities and recent updatesEnhanced Hierarchical GroupingHierarchical grouping is a new way to organize data into multiple levels within a single pivot table. This feature is especially useful when we need a more detailed breakdown without cluttering our workspace with multiple tables.<figure id="figure-31mothaa1tm7sufh48" class=""><img id="31mothaa1tm7sufh48" src="https://cdn.durable.co/blocks/0azcMAfyaWYDbXhdtezdsmJacrHm4FKF2nIalh2OudabJDFmFqje7FyYJ43Oj7iJ.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: 75% undefined%;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="75%" data-positiony="undefined%" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/0azcMAfyaWYDbXhdtezdsmJacrHm4FKF2nIalh2OudabJDFmFqje7FyYJ43Oj7iJ.png" data-mce-style="aspect-ratio: 16/9; object-position: 75% undefined%;"></figure><ul><li>What It Does:<ul><li>Allows grouping by multiple dimensions in a layered approach.</li><li>For example, we can first group data by Year, then by Region, and finally by Product Category.</li></ul></li><li>How to Use It :<ul><li>In Edit mode, we select the field we want to add to the hierarchy.</li><li>Click the “+” icon in the columns menu and choose to manage hierarchies.</li><li>We name our hierarchy and arrange the fields in the order we’d like them to appear</li></ul></li></ul><figure id="figure-wyz8shdjbm7sumpuq"><img id="wyz8shdjbm7sumpuq" src="https://cdn.durable.co/blocks/39y9yWbXeGoPMNGAtrv4RtfznPZGhwg3Lnik1YzH5XImNNdMnunw9HFMty2xLszC.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/39y9yWbXeGoPMNGAtrv4RtfznPZGhwg3Lnik1YzH5XImNNdMnunw9HFMty2xLszC.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><ul><li>Benefits:<ul><li>Saves time by automatically breaking down data into nested groups.</li><li>It offers a clear, layered view that lets us drill down into the details without overwhelming the table.</li></ul></li></ul>Automatic SubtotalsSubtotals are designed to give us intermediate sums for our grouped data, eliminating the need for manual calculations.<ul><li>How It Works:<ul><li>We can enable subtotals for any grouped field directly within the pivot table interface.</li><li>The table will then display intermediate sums for each group, as well as an overall total.</li></ul></li><li>Practical Benefits:<ul><li>We can quickly see the contribution of each segment to the overall total.</li><li>These are ideal for identifying trends, such as which region or product line is performing best. </li></ul></li></ul><figure id="figure-uwgnratctfm7suqigl"><img id="uwgnratctfm7suqigl" src="https://cdn.durable.co/blocks/36I00WCWDC8WobJLHnPEdq3Brq8l9WAHPRK7DyhLaIW8vDiwKq3C90531T1r6e3G.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/36I00WCWDC8WobJLHnPEdq3Brq8l9WAHPRK7DyhLaIW8vDiwKq3C90531T1r6e3G.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>Custom Total FormattingImproving the visual appeal of our pivot table is as important as the data itself. Sigma Computing now offers enhanced options for formatting totals and subtotals, so we can highlight key figures effectively.<ul><li>Customization Options:<ul><li>Adjust number formats (currency, percentage, etc.).</li><li>Modify font sizes, colors, and styles to draw attention to important figures.</li><li>Use conditional formatting rules to automatically highlight totals that exceed or fall below certain thresholds.</li></ul></li><li>Example Scenario:<ul><li>In a sales report, we might display the overall total in bold and with a larger font size, making it immediately noticeable against other data points.</li><li>This visual can help stakeholders quickly grasp overall performance at a glance.</li></ul></li></ul>These advanced functionalities, including hierarchical grouping, automatic subtotals, and custom total formatting, not only streamline our workflow but also provide a deeper understanding of the data—all while keeping the process user-friendly and visually engaging.<h3>Key Benefits of Pivot Tables in Sigma Computing</h3><ul><li>Real-Time Collaboration:<ul><li>Since Sigma Computing is cloud-based, multiple users can work on pivot tables simultaneously.</li><li>This ensures that decisions are based on the most current data.</li></ul></li><li>Easy Integration:<ul><li>Sigma integrates with various data sources, allowing us to easily pull in data from our cloud-based data warehouse and visualize it in pivot tables.</li></ul></li><li>Dynamic Updates:<ul><li>The data in our pivot table updates in real-time, so we don’t have to worry about out-of-date insights.</li></ul></li><li>Advanced Customization:<ul><li>From subtotals to conditional formatting, Sigma provides plenty of ways to fine-tune the appearance and functionality of our pivot tables.</li></ul></li><li>Beta Features:<ul><li>The latest additions, such as hierarchical grouping and enhanced subtotals, are continuously tested to make our data analysis even more intuitive and powerful.</li></ul></li></ul><h3>Conclusion</h3>Pivot tables are key tools for simplifying complex datasets, and Sigma Computing enhances them by making them more interactive and customizable. With powerful features like hierarchical grouping, subtotals, and custom formatting, we can create dynamic, insightful reports without the need for advanced coding or complex manual calculations.Whether we're analyzing sales, tracking performance, or conducting market research, Sigma's pivot tables provide a flexible, real-time way to summarize and present our data.By exploring these features, we can unlock new levels of data analysis and make more informed, decisions for our team or organization.

Pivot Tables in Sigma Computing

<h2>What is Embedded Analytics?</h2>Embedded analytics refers to the integration of data visualisations and business intelligence (BI) features within other applications, rather than using a separate BI tool. With Sigma's embedding options, we can directly integrate analytics into our company's internal portals, customer-facing applications, or operational workflows.Embedding analytics into our application provides several benefits -<ul><li>Simplified User Experience – Users get real-time insights without leaving the platform they are working in.</li><li>Increased Engagement – Interactive reports and dashboards encourage data-driven decision-making.</li><li>Custom Branding &amp; Experience – Maintain a consistent look and feel within our application.</li><li>Enhanced Security &amp; Governance – Control access through secure authentication methods.</li></ul>Sigma’s approach to embedding truly stands out. Whether you're looking to enhance user experience, provide real-time insights, or customise access controls, Sigma offers a wide range of embedding options tailored to different business needs.<h3>Embedding Methods in Sigma</h3>Sigma provides two main embedding options: Public Embedding and Secure Embedding. Let's explore both with real-world use cases.<figure id="figure-d1p94s3xmm7816thn"><img id="d1p94s3xmm7816thn" src="https://cdn.durable.co/blocks/1coqXU8AE5qgnk7MQGkEhELdrQztmaBrN59DF3PH59TuxfjlMmcfrFPBHK021te1.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/1coqXU8AE5qgnk7MQGkEhELdrQztmaBrN59DF3PH59TuxfjlMmcfrFPBHK021te1.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><h3>Public Embedding</h3>Public embedding is the simplest way to share analytics. It allows you to display content openly, without requiring authentication. If you’ve ever needed to showcase a live dashboard on an office screen or provide real-time metrics on a public webpage, this method is perfect.Example use case:<ul><li>Displaying live sales trends on an office TV screen</li><li>Sharing public reports on a company website </li></ul>Implementation:<ul><li>Create a dashboard in Sigma.</li><li>Click Share &gt; Embed and select Public Embed.</li><li>Copy the generated iframe or URL and insert it into your webpage or display screen.</li></ul><h3>Secure Embedding</h3>Secure embedding (previously known as "user-backed embedding") provides fine-grained control over who can see what. It ensures that data access is restricted based on the host application’s user authentication system. This method uses a signed URL generated via Sigma’s embed API, allowing for dynamic, user-specific access.From experience, this is the method most businesses prefer because it allows for personalized analytics without exposing sensitive data.Example use case:<ul><li>Embedding analytics in a customer portal where each user sees only their own data</li><li>Providing role-based dashboards within an internal enterprise tool</li></ul>Implementation:<ul><li>Generate a secure token on our server.</li><li>Use the Sigma Embed API to create a signed URL.</li><li>Embed the URL in your application using an iframe or JavaScript. <figure id="figure-pqnwuj8yc3m7816lqn"><img id="pqnwuj8yc3m7816lqn" src="https://cdn.durable.co/blocks/dSxSSjSg3eSfDLJP3hjgkcg5Z1ynk7zCdIr4Q8YbrJeDOpCYmtEuDhjQy1yr82je.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="null" data-positiony="null" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/dSxSSjSg3eSfDLJP3hjgkcg5Z1ynk7zCdIr4Q8YbrJeDOpCYmtEuDhjQy1yr82je.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure> </li></ul><h3>Advanced Data Security in Embeds</h3>When implementing secure embeds, Sigma provides advanced security controls:<ul><li>Column-Level Security – Restrict visibility of specific columns.</li><li>Row-Level Security – Display only the relevant rows based on user permissions.</li><li>Dynamic Connection &amp; Role Switching – Users see data based on their assigned roles.</li><li>&nbsp;OAuth Integration – Enable seamless authentication using your existing identity provider. </li></ul><h3>Additional Embedding Configurations</h3>Sigma offers additional features to enhance the embedded analytics experience:<ul><li>JavaScript Actions &amp; Events – Add interactivity and respond to user actions.</li><li>Responsive Design – Ensure dashboards adjust to different screen sizes.</li><li>Version Tagging &amp; Localization – Control versions and support multi-language embeds.</li><li>Bookmarks &amp; Saved Views – Allow users to save custom views of reports.</li></ul><h3>Key Takeaways</h3>Embedded analytics in Sigma provides a simple way to integrate powerful BI capabilities into your applications. Whether you need public embedding for general dashboards or secure embedding for user-specific insights, Sigma offers flexible solutions to enhance your product's data experience.By using Sigma's embedding capabilities, businesses can drive better engagement, improve data accessibility, and ensure secure, real-time decision-making for their users.

Embedded Analytics in Sigma Computing

<h2>What is Data Governance?</h2>In our last blog, we discussed how cloud costs can quickly add up and shared some easy ways to keep spending under control. We covered tips like using the right-sized resources and making the most of tools like Databricks and Snowflake to avoid paying for things you don’t need. Now, we’re moving on to an equally important topic: data governance.In today's data-driven world, the ability to access, manage, and analyze data has become a crucial competitive advantage for businesses. However, this also means that organizations must maintain strong data governance to ensure security, compliance, and privacy. With an ever-growing volume of data and a complex regulatory environment, CEOs and CIOs face increasing pressure to manage and protect their data assets effectively.Data governance is more than just setting policies and rules; it is about creating a robust framework that supports data management across an organization. This blog will discuss the critical aspects of data governance, the challenges organizations face, and how they can build an effective governance strategy to ensure data is secure, compliant, and private.<h3>What is Data Governance, and Why Does It Matter? </h3>Data governance involves establishing policies, procedures, and standards that govern data management across an organization. It ensures that data is accurate, consistent, secure, and used responsibly. Effective data governance is essential for several reasons:<ul><li>Data Quality and Trust: Data quality, accuracy, and consistency are critical to building stakeholder trust. Poor data quality can lead to bad decisions, reputational damage, and financial loss.</li><li>Regulatory Compliance: With stringent data privacy laws like GDPR, CCPA, and HIPAA, organizations must ensure they manage data in compliance with regulatory requirements to avoid hefty fines and legal repercussions.</li><li>Data Security and Privacy: As data breaches become more common, protecting sensitive information is crucial. Data governance helps organizations manage access, protect against unauthorized access, and maintain privacy.</li><li>Efficient Data Management: Governance ensures that data is organized and accessible, making it easier for teams to access the data they need when they need it without compromising security.</li></ul><h3>Challenges in Implementing Data Governance </h3>While data governance is essential, implementing an effective program is challenging. Organizations often face several obstacles when trying to build a governance framework that aligns with their business needs:<ul><li>Lack of Clarity on Data Ownership: Defining data ownership and stewardship roles can be challenging, especially in organizations where data is spread across multiple departments and platforms.</li><li>Data Silos: Many organizations suffer from data silos where data is stored in isolated systems, making it difficult to access and manage centrally. This results in inconsistencies, duplication, and compliance risks.</li><li>Complex Regulatory Landscape: Navigating multiple, often conflicting, regulatory requirements can be daunting, especially for global organizations operating across different jurisdictions.</li><li>Balancing Security with Accessibility: Ensuring data is secure and accessible to those who need it can be tricky. Too many controls can limit accessibility, while too few can increase risks.</li><li>Resource Constraints: Data governance initiatives require dedicated resources, including time, budget, and skilled personnel. For many organizations, especially smaller ones, these resources may be limited.</li></ul><h3> Building an Effective Data Governance Framework</h3>Organizations need a well-defined and scalable data governance framework to overcome these challenges that align with their business objectives. Here are the critical components of an effective data governance strategy:<ul><li>Establish Clear Roles and Responsibilities: Defining roles is crucial for accountability in data governance. This involves assigning data owners, stewards, and custodians responsible for data quality, security, and compliance.</li><li>Develop a Comprehensive Data Governance Policy: A data governance policy outlines the rules and guidelines for managing data across the organization. It should cover data classification, quality standards, access control, data sharing, and retention policies.</li><li>Implement Data Lineage and Metadata Management: Data lineage provides a clear view of where data comes from, how it flows through the organization, and how it is transformed. Metadata management involves capturing and managing data about the data itself.</li><li>Automate Compliance Monitoring: Compliance monitoring should be a continuous process rather than a one-time effort. Automating compliance checks can help organizations stay ahead of regulatory changes and reduce the risk of non-compliance.</li><li>Enable a Data-Driven Culture: A successful data governance program requires a cultural shift toward data-driven decision-making. This involves educating employees about the importance of data governance, promoting data literacy, and encouraging the responsible use of data.</li><li>Leverage Technology for Effective Data Governance: Technology plays a crucial role in data governance by automating processes, improving data quality, and enhancing security. Data catalogs, quality solutions, and lineage tools are essential to a modern data governance framework. </li></ul><figure id="figure-k4bncpw1eam6xwswm1"><img id="k4bncpw1eam6xwswm1" src="https://cdn.durable.co/blocks/53A1idw6SvF2oH0z0i08COvAwUmQG5MPltndXNYSoE66t5aK4o9LHmEUcHiuWEzS.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/53A1idw6SvF2oH0z0i08COvAwUmQG5MPltndXNYSoE66t5aK4o9LHmEUcHiuWEzS.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><h3>Ensuring Privacy and Security in a Modern Data Stack</h3>Ensuring privacy and security has become more challenging with the increasing volume of data being processed and stored. Here are some strategies to safeguard your data in a modern data stack:<ul><li>Data Masking and Encryption: Data masking involves obfuscating sensitive data to protect it from unauthorized access. Encryption ensures that data is protected in transit and at rest. Example: For a financial services company handling sensitive customer information, data masking can protect data in non-production environments, while encryption ensures data security in transit and storage.</li><li>Access Management and Monitoring: Implement role-based access control (RBAC) and least privilege principles to limit access to sensitive data. Regularly monitor data access logs to detect unauthorized access or anomalies. For example, an e-commerce company can use RBAC to restrict access to customer data to only authorized personnel and set up alerts for unauthorized access attempts.</li><li>Regular Audits and Reviews: Conduct regular data audits and reviews to ensure compliance with data governance policies and regulations. Audits help identify gaps, improve processes, and ensure continuous compliance. For example, a healthcare provider can conduct periodic data audits to ensure compliance with HIPAA regulations and avoid potential breaches or fines. </li></ul><figure id="figure-nrndb4sjgdm6xwtntj"><img id="nrndb4sjgdm6xwtntj" src="https://cdn.durable.co/blocks/1fVo9yAKOpK1PMwA0hOAjBeLzqOBVBpydyBVdBcbS5n6BcwBz0Byxgz3xqm1ghXL.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/1fVo9yAKOpK1PMwA0hOAjBeLzqOBVBpydyBVdBcbS5n6BcwBz0Byxgz3xqm1ghXL.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-acmnmhdopgm6xwu6j9"><img id="acmnmhdopgm6xwu6j9" src="https://cdn.durable.co/blocks/35Xzuu25kuEdkj2Iq5ugmlClND1akxkYnM2BZ8VB10fWJcL6zCtuZmt5MWsKlyVj.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/35Xzuu25kuEdkj2Iq5ugmlClND1akxkYnM2BZ8VB10fWJcL6zCtuZmt5MWsKlyVj.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-paproz9xdm6xwunuz"><img id="paproz9xdm6xwunuz" src="https://cdn.durable.co/blocks/103zexIe89gvxmJTn3wxfl0sgdQPMRlc89RqdBCP6qlaFKrIL6sho7AtCGVvmfDo.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/103zexIe89gvxmJTn3wxfl0sgdQPMRlc89RqdBCP6qlaFKrIL6sho7AtCGVvmfDo.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-p4x1aar6hbm6xwv5qn"><img id="p4x1aar6hbm6xwv5qn" src="https://cdn.durable.co/blocks/27qKxJ7AnxTBZ20o0og5mP7n9vlEWEtXU0T4SVv24k1CKgXm7AqtjRvOePQN2gYU.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/27qKxJ7AnxTBZ20o0og5mP7n9vlEWEtXU0T4SVv24k1CKgXm7AqtjRvOePQN2gYU.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-se8sqctffm6xwvocg"><img id="se8sqctffm6xwvocg" src="https://cdn.durable.co/blocks/8JtmESf4IhF93P1YRbir5xGdioZ2sDNHnGwpgDk9Nnevvez7Rd1eWE6wRS4Si62W.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionX="undefined" data-positionY="undefined" data-fullWidth="false"/></figure><h3>Conclusion: A Strategic Approach to Data Governance</h3>Data governance is a critical component of a successful data strategy. Organizations can ensure data security, compliance, and privacy by implementing a comprehensive data governance framework while empowering their teams to make data-driven decisions. CEOs and CIOs should focus on building a governance program that aligns with business goals, embraces a data-driven culture, and continuously adapts to the evolving data landscape.At Kitesight, we understand the complexities of managing data in a modern environment. Our expertise can help you build a robust data governance framework that ensures your data is secure, compliant, and aligned with your business objectives. Let's work together to turn data governance from a challenge into a strategic advantage.

Data Governance: Ensuring Data Security, Compliance, and Privacy

<h2>Rising Costs: How to Keep Your Cloud Spending Under Control</h2>In today's digital age, cloud services have become essential for businesses aiming to stay competitive and efficient. However, as more companies rely on the cloud, managing the rising costs has become a significant challenge for CEOs and CFOs. Cloud services offer immense flexibility and power but can lead to unexpected and hefty bills without careful management. This blog will explore practical ways to keep your cloud spending under control, helping you make the most of your investments without overspending.<h3>Understanding Where Your Money Goes</h3>Cloud computing offers scalability and flexibility that traditional data centers can't match. But with these benefits come complexities that can make tracking and controlling costs hard. According to Gartner, companies that don't manage their cloud usage effectively could overspend by 20% to 50% in the coming years. For a company with a $10 million cloud budget, this could mean wasting $2 to $5 million annually.This is a severe concern for CEOs and CFOs. Understanding where your cloud budget is going and finding ways to optimize spending without sacrificing performance or efficiency is essential.<h3>Optimizing Costs: The Key to Control</h3>The first step to managing cloud costs is optimization. This involves examining how your company uses cloud resources and identifying areas where you might overspend. Here are some simple strategies to get started:<ul><li>Right-Sizing Your Resources: Right-sizing means matching your cloud resources to your needs. Many companies overestimate the computing power or storage they need, leading to wasted capacity. You can reduce waste and save money by aligning resources with actual usage.</li></ul>Example: Imagine your company runs several virtual machines (VMs) on AWS, but some only use 10-15% of their capacity. By downsizing or consolidating these VMs, you could significantly reduce your monthly bill without affecting performance.<ul><li>Using Reserved Instances and Spot Instances: Reserved Instances (RIs) let you commit to using specific resources over a set period, often at a much lower cost—up to 75% less than on-demand pricing. Spot Instances allow you to use spare cloud capacity at a steep discount, sometimes up to 90% cheaper.</li></ul>Example: If your company needs extra computing power for occasional tasks, like running end-of-month reports, using Spot Instances can provide the capacity you need without the high costs.<ul><li>Implementing Auto-Scaling: Auto-scaling automatically adjusts your resources based on demand, ensuring you only pay for what you use. This can lead to significant savings, especially during periods of fluctuating demand.</li></ul>Example: During a big product launch, your website might see a surge in traffic. Auto-scaling ensures you have enough resources to handle the spike but scales down when traffic normalizes, preventing unnecessary spending.<figure id="figure-oqqkcbf98nm6q2iqkt"><img id="oqqkcbf98nm6q2iqkt" src="https://cdn.durable.co/blocks/3bP1GnzDYQiaR3MZGXrFIYLrIl9MUieWmu22C0eoa5HHke2xAmkSswxxXPPRwmX5.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/3bP1GnzDYQiaR3MZGXrFIYLrIl9MUieWmu22C0eoa5HHke2xAmkSswxxXPPRwmX5.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><figure id="figure-v3uorbk89sm6q2jpjm"><img id="v3uorbk89sm6q2jpjm" src="https://cdn.durable.co/blocks/26B4hGsWrpk13rdIWCOToT8uYzBiGewbsrPxBpCk2bx9ZB9TbHU0hPhP8o0WUlLc.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/26B4hGsWrpk13rdIWCOToT8uYzBiGewbsrPxBpCk2bx9ZB9TbHU0hPhP8o0WUlLc.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure> <h3>Reducing Data Transfer and Storage Costs</h3>Storing and transferring data can be a significant part of your cloud expenses. Here’s how to manage these costs effectively:<ul><li>Optimizing Data Storage: Instead of repeatedly querying your databases for the same information, consider storing frequently accessed data in more cost-effective cloud storage solutions like AWS S3 or Google Cloud Storage. This approach is generally cheaper than constantly processing the same data.</li></ul>Example: If your company regularly accesses customer data, storing this information in a low-cost storage service and updating it periodically could reduce query costs by 40-50% annually.<ul><li>Minimizing Data Transfer Costs: Moving data across cloud regions or to external systems can add up quickly. To reduce these costs, consider these strategies:<ul style="list-style-type: square;" data-mce-style="list-style-type: square;"><li>Keep Data Within the Same Region: Data transfers within the same region are often cheaper than across areas.</li><li>Use Data Caching: By caching frequently accessed data, you can reduce the need for constant data transfers, lowering costs.</li></ul></li></ul>Example: A company with global operations could save significantly by storing data centrally in one region and caching it locally rather than constantly transferring it across regions.<figure id="figure-plppo1mcpcm6q4f2uf"><img id="plppo1mcpcm6q4f2uf" src="https://cdn.durable.co/blocks/aiJGsOwfJLZSqkgDShnXjJrAPfnnLs6wmHXibVyOONn5pXkMWlk7klPtwz9c4dn2.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/aiJGsOwfJLZSqkgDShnXjJrAPfnnLs6wmHXibVyOONn5pXkMWlk7klPtwz9c4dn2.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure><h3>Monitoring and Forecasting Cloud Costs</h3>Continuous monitoring and forecasting are essential to keeping cloud costs in check. CEOs and CFOs should see cloud spending and understand how it aligns with business goals.<ul><li>Real-Time Cost Monitoring: Tools like AWS Cost Explorer, Google Cloud's Cost Management, and Azure Cost Management provide real-time insights into your cloud spending. These tools help you track costs, identify trends, and make informed decisions.</li></ul>Example: A company with seasonal business fluctuations could use these tools to adjust cloud usage dynamically, ensuring that costs align with current business needs.<ul><li>Setting Budgets and Alerts: Setting budgets and alerts when spending exceeds certain thresholds can prevent overspending. Regular reviews of cloud bills help catch unexpected charges and identify areas for cost savings.</li></ul>Example: A startup could set a monthly cloud budget and receive alerts when spending reaches 80% of the budget, giving the team time to investigate and adjust usage before exceeding the limit.<ul><li>Forecasting Future Costs: Predictive analytics tools can help forecast future cloud expenses based on current usage patterns and anticipated growth. This allows CFOs to plan and allocate resources effectively.</li></ul>Example: A logistics company planning to expand its services could use forecasting tools to estimate the additional cloud costs, ensuring the budget aligns with its growth strategy.<figure id="figure-mp90ktc4lhm6q4j2pp"><img id="mp90ktc4lhm6q4j2pp" src="https://cdn.durable.co/blocks/4WNpdl7KlUfRZZDnPPCPRVGFnmm2S71o1wDdsOSn7q0qgaFUmBaLiTzJV3vAClhT.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/4WNpdl7KlUfRZZDnPPCPRVGFnmm2S71o1wDdsOSn7q0qgaFUmBaLiTzJV3vAClhT.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>Now let's look at how we can implement the practices mentioned above for the modern cloud data warehouse below:<h3>Managing Costs with Databricks and Snowflake</h3>Tools like Databricks and Snowflake are essential for data analysis but can become costly if not used wisely. Here’s how to manage these expenses:<ul><li>Understanding the Pricing Models: Databricks charges based on computing time, while Snowflake charges for storage and compute credits. Understanding these models is crucial for predicting and managing costs.</li></ul>Example: A company using Databricks for data analysis could save money by scheduling less urgent tasks during off-peak hours when computing costs are lower.<figure id="figure-gvayftbhy9m6q4o0wc"><img id="gvayftbhy9m6q4o0wc" src="https://cdn.durable.co/blocks/399tcdq3CuVk7cuNMehlotBsZlVTnkieWcEaQkRpwwtAcu712VXZ9B0Wq6WDcTSV.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/399tcdq3CuVk7cuNMehlotBsZlVTnkieWcEaQkRpwwtAcu712VXZ9B0Wq6WDcTSV.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure> <ul><li>Optimizing Your Workloads:<ul style="list-style-type: square;" data-mce-style="list-style-type: square;"><li>Databricks: Use job clusters for batch processing and interactive clusters for development. This ensures you're not overpaying for resources you don't need.</li><li>Snowflake: Compress data and use Snowflake’s micro-partitioning feature to reduce storage costs and speed up queries.</li></ul></li></ul>Example: By compressing old data in Snowflake, a company could reduce storage costs by 20% and improve query speed, allowing faster data insights at a lower price.<ul><li>Storing Data Efficiently: Databricks and Snowflake offer cheaper storage options. Archiving old or infrequently accessed data in lower-cost storage can lead to significant savings.</li></ul>Example: A media company could save up to 60% on storage costs by moving older, less frequently accessed content to lower-cost storage options.<figure id="figure-015lxvigm6q5pcd6"><img id="015lxvigm6q5pcd6" src="https://cdn.durable.co/blocks/1dimtKFfakdQvJYd7NTFfkEVq4rEBoaEFO0L3OYaYPh9FXuQeMaEDBNwVCucsy8W.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionX="undefined" data-positionY="undefined" data-fullWidth="false"/></figure><h3>Conclusion: Smart Cloud Cost Management</h3>For CEOs and CFOs, managing cloud costs is not just about cutting expenses—it's about ensuring that investments in cloud technology drive value for their businesses. You can control your cloud spending while supporting your company's growth and innovation by implementing strategies like right-sizing, optimizing storage and data transfers, and continuously monitoring costs.At Kitesight, we’re here to help you navigate the complexities of cloud cost management. Our expertise can guide you in optimizing your cloud strategy, reducing unnecessary expenses, and focusing on what matters most—driving your business forward.Stay tuned for our next blog, where we’ll learn more about the critical topic of Data Governance. We'll explore how to secure your data, ensure compliance, and maintain privacy, enabling your organization to manage data more effectively and confidently in today’s complex digital landscape. Don't miss it!

Optimizing Your Cloud Platform Costs

<h2>What if You’re Already on a Modern Data Stack and Still Struggling?</h2><h3>Introduction </h3>In our blogs, we talked about the benefits of moving from older data systems to a modern data stack—how using the latest tools and technologies can make your business more efficient and flexible.But the journey doesn’t end with modernization. Even with the advantages, modern data stacks come with their own set of challenges. In today’s fast-paced world, many organizations have switched to cloud infrastructure, advanced analytics, and data management tools. Yet, despite these upgrades, companies still face significant hurdles. Rising costs, complex compliance, tech overload, and data duplication are just a few issues that can prevent you from making the most of your modern data stack.Why do these issues happen? Often, the tools don't match the organization’s specific needs, workflows aren’t optimized, or there’s a lack of proper oversight. These problems can build up, leading to higher costs, data silos, compliance risks, and tool overload. That’s why it’s essential to implement and regularly monitor, refine, and align your data stack with your business goals.At Kitesight, we understand that transitioning to a modern data stack is the first step. The real work begins with ensuring that your stack is optimized, cost-effective, and aligned with your business needs.In this blog post, we’ll explore how Kitesight can help you tackle these challenges and fully unlock the potential of your modern data solutions.1. Rising Costs: Keeping Your Cloud Spending Under Control <figure id="figure-gnwfj8ilbmm6dqefgb"><img id="gnwfj8ilbmm6dqefgb" src="https://cdn.durable.co/blocks/26kr5yn3mpT0M6akXMkLw4FclN62GpQRU63e1HgJPRxzWkXXulOGd25SATY2Y33B.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/26kr5yn3mpT0M6akXMkLw4FclN62GpQRU63e1HgJPRxzWkXXulOGd25SATY2Y33B.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure> Cloud services are powerful and flexible but can become expensive if not managed well. Many companies incur higher bills than expected as storage, computing, and processing costs increase.How Kitesight Can Help:<ul><li>Cost Optimization: We help you understand your cloud usage, find areas where you're spending too much, and apply cost-saving strategies. We use tools like Reserved Instances, autoscaling, and cost-monitoring dashboards to ensure you only pay for what you need.</li><li>Resource Management: We ensure that your cloud resources, like storage and computing power, are used efficiently. This helps prevent overspending and keeps costs down.</li><li>Custom Alerts &amp; Reporting: We set up alerts and reports to keep you updated on your cloud spending so you don’t get surprises at the end of the month.</li></ul>For example, we recommend storing frequently accessed data in the cloud instead of querying data multiple times, which can increase costs. This cost-effective method speeds up data retrieval, ultimately saving money.2. Data Governance: Keeping Your Data Safe, Compliant, and Private<figure id="figure-yfej0vb74am6dqiph2"><img id="yfej0vb74am6dqiph2" src="https://cdn.durable.co/blocks/321jAh1KUEYNy6Cy3Cqa5rqCOL0tpRDBi9I7x564b6hBMH2IffG9Z1myY9KBDWLq.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/321jAh1KUEYNy6Cy3Cqa5rqCOL0tpRDBi9I7x564b6hBMH2IffG9Z1myY9KBDWLq.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>Keeping data secure and following the rules can become complicated as data grows. It’s essential to protect your data, make it accessible to the right people, and stay compliant with regulations.How Kitesight Can Help:<ul><li>Comprehensive Data Governance Framework: We create robust frameworks that match industry standards. We ensure your data is secure, well-organized, and accessible to authorized users.</li><li>Automated Compliance Monitoring: We help you automate compliance checks to ensure your data meets all necessary regulations. This reduces the risk of fines, breaches, and reputational damage.</li><li>Data Classification &amp; Access Control: We help you classify data by sensitivity and set up controls to ensure that only the right business stakeholders can access the necessary information. This makes it easier to stay secure and compliant.</li></ul> For example, by classifying and restricting access to sensitive customer data, we help you prevent unauthorized access and ensure compliance with regulations like GDPR, reducing risk and protecting your reputation.3. Tech Stack Optimisation: Simplifying Your Toolset for Better Efficiency With so many data tools and platforms available, feeling overwhelmed is easy. Managing too many tools can lead to inefficiencies and a lack of clear direction in your data strategy.How Kitesight Can Help:<ul><li>Tool Consolidation: We review your current tools and find ways to consolidate them. By reducing the number of tools and focusing on the best ones, we help you streamline your operations and reduce complexity.</li><li>Integration Expertise: We ensure your tools work well together, creating a smooth and efficient data environment. Our expertise in platforms like dbt, Snowflake, and Databricks ensures that everything runs as a cohesive system.</li><li>Operational Efficiency: By optimizing workflows and eliminating unnecessary tools, we help you work more efficiently by reducing tech debt. This allows your team to focus on key business insights rather than managing the stack.</li></ul> For example, by consolidating your data tools and integrating platforms like dbt, Snowflake, and Databricks, we simplify your operations and enhance efficiency, enabling your team to focus more on insights and less on managing disparate technologies.4. Auditing: Making Sure Your Data is Traceable<figure id="figure-oipirw8nnpm6dql2qz"><img id="oipirw8nnpm6dql2qz" src="https://cdn.durable.co/blocks/2455EEekInAlxODTrDYqPqI4tfg8Q0fAGUZT9Rms5GmkRVUOGrj3HFZUwx6ou22A.png" alt="" title="" class="object-cover bg-gray-100 rounded-lg md:rounded-xl lg:rounded-2xl" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionX="undefined" data-positionY="undefined" data-fullWidth="false"/></figure>Auditing is critical to keeping data accurate and following the rules. But tracking where data comes from, how it’s changed, and where it goes can be challenging, especially in complex data systems.How Kitesight Can Help:<ul><li>Data Lineage Tools: We use advanced tools, like DataHub, to map out the data flow within your systems. These tools help track and document every step in your data’s journey, making auditing easier and ensuring compliance.</li><li>Audit Trails: We create detailed logs of who accessed data, what changes were made, and where the data went. This transparency is crucial for both internal checks and external audits.</li><li>Compliance Reporting: We help you generate automatic compliance reports, making it easier to prove you follow the rules during audits.</li></ul> For example, we can track where your data goes and who accesses it, making it easy to comply with regulations like GDPR and pass audits with automated reports.<h3>&nbsp; Getting the Most Out of Your Modern Data Stack</h3> Switching to a modern data stack is a big step but comes with challenges. Whether you need help managing costs, keeping data secure, handling audits, simplifying your tools, or reducing data duplication, Kitesight is here to help. We ensure your data stack is modern, optimized, efficient, and aligned with your business goals.At Kitesight, we turn data challenges into opportunities for growth. Let us help you unlock the full potential of your data stack and drive your business forward with confidence. Discover how Kitesight can make your data work for you.In our next blog, we'll focus on one of the biggest concerns for companies today—keeping cloud costs under control. We'll look at easy ways to lower your cloud spending, like storing data smartly and avoiding extra charges. Stay tuned to learn how to manage your cloud budget without missing out on the benefits of your modern data stack. &nbsp; &nbsp;

Blog

Working with Geospatial Data in Sigma Computing

Pivot Tables in Sigma Computing

Embedded Analytics in Sigma Computing

Data Governance: Ensuring Data Security, Compliance, and ...

Optimizing Your Cloud Platform Costs

Optimizing Modern Data Stack