Periodically, organizations must tackle performance challenges in report execution. Power BI offers the Performance Analyzer tool, aiding in issue resolution and process optimization.
Imagine creating reports for your organization's Sales team. You've connected to the Sales team's SQL database using DirectQuery, importing data from various tables. As you develop initial visuals and filters, you observe variations in query speed and processing time across different tables and filters.
Optimise performance in PowerQuery
Power Query performance is influenced by the efficiency of the underlying data source. Given the diverse range of data sources available in Power Query, optimizing performance requires specific tuning techniques tailored to each source. For instance, when extracting data from Microsoft SQL Server, it is advisable to adhere to the product's performance tuning guidelines. Effective SQL Server tuning involves practices such as index creation, hardware upgrades, execution plan tuning, and data compression. While these details are presented as an illustrative example, they serve to familiarize users with their data source, facilitating enhanced performance when utilizing Power BI and Power Query.
An essential approach employed by Power Query to leverage optimal data source performance is known as Query Folding.
Query Folding
Query folding in Power Query Editor is a performance-enhancing feature for Power BI reports. This mechanism tracks the transformations made in Power Query Editor as native queries or straightforward Select SQL statements in real-time during the editing process. The primary goal is to execute these transformations directly on the original data source server, preventing any strain on Power BI computing resources.
Utilizing Power Query, you can load data into Power BI and leverage Power Query Editor for various data transformations, including tasks like renaming or deleting columns, appending, parsing, filtering, and grouping your data.
Query folding offers advantages such as enhanced efficiency in data refreshes and incremental updates. Utilizing query folding during data table imports allows Power BI to optimize resource allocation and accelerate data refreshing by avoiding the need to execute each transformation locally.
Moreover, query folding seamlessly integrates with DirectQuery and Dual storage modes. For data sources employing DirectQuery and Dual storage, query folding becomes an automatic feature, leveraging the back-end server's processing capabilities to establish a direct connection. This means that query folding is readily available for use. The condition for query folding initiation is met when all transformations can be condensed into a single Select statement.
The following scenario shows query folding in action. In this scenario, you apply a set of queries to multiple tables. After you add a new data source by using Power Query, and you're directed to the Power Query Editor, you go to the Query Settings pane and right-click the last applied step, as shown in the following figure.
If the View Native Query option lacks visibility (not in bold), it indicates that query folding cannot be applied at this stage. To enable query folding, you need to trace back through the Applied Steps until you find the step where View Native Query is available (in bold). This step will unveil the native query responsible for transforming the semantic model.
Certain transformations do not support native queries, including:
Adding an index column
Merging and appending columns from tables with distinct sources
Changing the data type of a column
A helpful guideline is that if you can express a transformation as a Select SQL statement with operators like GROUP BY, SORT BY, WHERE, UNION ALL, and JOIN, query folding is feasible.
While query folding is a performance optimization method during data retrieval, import, and preparation, an alternative approach is query diagnostics.
Query Diagnostics
Query diagnostics is a valuable tool for analyzing query performance in various tasks such as loading, transforming data, refreshing data in Power Query, and executing SQL statements in Query Editor. It helps identify potential bottlenecks in these processes.
To initiate query diagnostics in Power Query Editor, navigate to the Tools option in the Home ribbon. Once you're prepared to commence data transformation or other edits, choose Start Diagnostics in the Session Diagnostics section. Ensure to conclude the diagnostic process by selecting Stop Diagnostics.
Selecting Diagnose Step shows you the length of time that it takes to run that step, as shown in the following image. This selection can tell you if a step takes longer to complete than others, which then serves as a starting point for further investigation.
This tool is useful when you want to analyze performance on the Power Query side for tasks such as loading semantic models, running data refreshes, or running other transformative tasks.
Other techniques to optimise performance
Other ways to optimize query performance in Power BI include:
Process as much data as possible in the original data source. Power Query and Power Query Editor allow you to process the data; however, the processing power that is required to complete this task might lower performance in other areas of your reports. Generally, a good practice is to process, as much as possible, in the native data source.
Use native SQL queries. When using DirectQuery for SQL databases, such as the case for our scenario, make sure that you aren't pulling data from stored procedures or common table expressions (CTEs).
Separate date and time, if bound together. If any of your tables have columns that combine date and time, make sure that you separate them into distinct columns before importing them into Power BI. This approach will increase compression abilities.
SWETA SARANGI
20.02.2024