Release Notes for Astera Data Virtualization (Beta)

Today, we officially kick off the beta release of our new solution, Astera Data Virtualization. This release continues our commitment to deliver new solutions with improved capabilities to help bring some order to the complexity of enterprise data management and enable easier and faster access to data in a self-service manner.

Astera Data Virtualization is designed to help businesses serve up integrated data on demand. It enables enterprises to break down data silos by integrating data from disparate sources and presenting it as a virtual database, thereby shielding the end-user from the complexities of connecting to each data source individually, while limiting physical data movement and optimizing the use of developer resources.

Here’s a roundup of the key capabilities that the product features:

Integrate Any Data Source without Data Replication

With the new solution, we are shipping the capability to integrate and present disparate data sources as a virtual database to business users. This will help minimize data replication and serve filtered datasets according to the business requirements.

The new solution features the same intuitive integrated development environment (IDE) that Centerprise boasts, along with connectivity to a range of data sources.

Connectors

Databases	Access popular databases like SQL Server, Oracle, PostgreSQL, MySQL, MS Access, Sybase, IBM DB2, and others via Open Database Connectivity (ODBC) and Object Linking and Embedding, Database (OLE DB)
Cloud Databases	Use cloud databases as a data source in your virtual data model with connectivity to Amazon Redshift and Microsoft Azure*
Packaged Applications	Salesforce, Microsoft Dynamics CRM
BI and Analytics Tools	PowerBI and Tableau*
Data Warehouses	Teradata, Netezza
Structured, Semi-Structured, and Unstructured File Formats	Excel, delimited files, fixed-length text files, XML, JSON, TXT, RTF, DOC, DOCX, PDF, PDF Forms, COBOL files and copybooks, EDI formats, including ANSI X12, EDIFACT, HIPAA, and HL7
Web Services	REST, SOAP, FTP, SCP, HDFS, AS2, ADO. Net, emails, and file system

 

Integrated Development Environment (IDE)

Drag-and-drop Interface	Add new data sources, apply transformations, or define destinations in simple drag-and-drop actions
Code-free, Graphical Development Environment	Model data, define relationships between entities, build transformations, and more in a graphical, code-free manner
Project Explorer	Keep your data virtualization projects organized with a Project Explorer that allows easy access to schedules, parameters, virtual data models, and shared connections

 

Data Source Browser

Access Database Sources	Easily navigate all your database sources and their structure via Data Source Browser
Query Data	Query a given database directly from the Data Source Browser by writing SQL queries
Export Data	Export data from a database to an Excel or delimited file right from the Data Source Browser with a single click

 

Create and Share Custom Data Views with Business Users

Data replication takes time and money, particularly when there are evolving business requirements and massive volumes of disparate data to handle. With Astera Data Virtualization, we are delivering the capability to create and deliver filtered data views to data consumers in the form of a virtual database. The virtual database can be used in a dataflow for further querying using the multi-table query source and Astera Query Language (AQL) or exposed via ADO.NET or REST APIs.

Astera Data Model Browser

Access Virtual Databases	Quickly and easily browse through the virtual databases deployed on a server and consume them in a dataflow for transformation or data transfer
Query a Virtual Database	Drag and drop individual tables onto the designer or use multi-table query source to use a virtual database as a source in a dataflow

ADO.NET/REST APIs

Expose Virtual Data Models over REST	Publish virtual data models in the REST format. REST APIs for create, read, update, and delete (CRUD) operations are supported.*
Expose Virtual Data Models over ADO.NET	Expose virtual data models as a service, available to clients from across the intranet and internet via ADO.NET. These services are accessible over HTTPS using URIs.

Multi-Table Query Source

Query a Virtual Database	Use the multi-table query source to use a virtual database as a source. Particularly useful for addressing complex business requirements that involve combining data spread out in more than one tables
Query Filtered Data	Apply different filters, such as less than or greater than, is null or not null, is empty string, and others to fetch filtered datasets from a virtual database

Astera Query Language

Automatically Create Joins	Write AQL statements in the query window or automate complex joins using the multi-table query source
Connect to Non-SQL Sources	Connect to non-SQL sources, such as Excel, PDF, delimited file sources, email, and more
Edit Auto-Generated AQL Commands	Edit the auto-generated AQL commands easily to retrieve filtered datasets from a virtual database
AQL Commands	Use common commands, such as SELECT and ORDER BY commands to fetch, join, and sort data from a database table/s

Query Optimization with Caching and Indexing

For faster processing of queries performed on a virtual database, we are rolling out two key features – caching and cache indexes. With caching, users can create a temporary copy of data on their local server and avoid going back to the original data sources or making repetitive API calls every time they want to fetch data. Adding custom indexes to the cache tables will help further speed up data retrieval.

Caching and Cache Indexes

Scheduled Cache Refresh	Specify hourly, daily, weekly, or monthly frequency to refresh the cache at fixed intervals
One-time Caching	Create a cache repository of sources whose data does not change frequently, such as lookup tables, using the one-time caching option
Continuous Caching	Refresh the cache continuously throughout the day or during specified hours using this option
Incremental Refresh	Update only the most recent data using the incremental cache refresh option by specifying an audit field, such as date or time, version number, or a key field
Full Refresh	Refresh the entire cache once, at specified intervals, or continuously
Event-Based Refresh	Refresh cache repository when a file is dropped in the specified folder or shared drive. You can also define the files to watch for by specifying the file extension, such as .txt or .xlsx.
Submission Caching	Watch an online location on Astera Cloud Portal and update the cache whenever a new file is submitted by a supplier
Cache Indexes	Create cache database indexes to improve the speed of data retrieval. Supports clustered, unique, unique constraint, and primary key indexing methods.

Build, Test, and Deploy Data Virtualization Projects in Teams

Astera Data Virtualization also offers improved source control and project archiving features, enabling users to keep track of changes and keep a backup of each project to enhance reusability and minimize rework. This not only facilitates auditing, but also expedites re-deployment of existing data virtualization projects.

Source Control

Source Control Explorer	Use Source Control Server to browse different data virtualization projects and their versions
Source Control Server	Connect to a source control system, such as VSTS or Git, to simplify collaboration and keep track of every change made to a data virtualization project
Pending Changes	Organize, manage, and get details about what you have changed in a project with the ability to see and check in or undo pending changes
Compare Versions	Compare two different data virtualization projects and view the differences between the contents of each project
Workspaces	Develop and test your data virtualization projects in isolation on a local or server workspace

Project Archive

Create Project Archive (*.Car)

Maintain a backup of all virtual data models using the archive functionality that automatically generates a *.Car file every time a Virtual Data Model is re-deployed after changes

Secure and Monitor Access to Virtual Data Models

With a centralized deployment monitor, Astera Data Virtualization allows simpler and more efficient management of virtual data model deployments. Users can keep track of important health indicators, as well as detailed information about entity caching in the virtual deployment monitor. In addition, with user authentication and authorization features, administrators can manage access to all deployed virtual databases on a server*.

Virtual Deployment Monitor

Deployment Log	Get information about the deployment of a virtual data model, including copying of the archive file to a staging directory, creation and initial population of the cache tables, execution of cache dataflows, and changes to deployment or cache databases in case of redeployment
Entity Cache Info	Access caching details for all the entities comprising your virtual data model at a single place, including the caching schedule, cache status, last update, record count, and scheduled cache jobs
Job Progress	Get complete details about the jobs that were run as part of a virtual data model deployment along with their progress and SQL script, if any.

User Authentication

Bearer-token Authentication	Make access distinctions and authenticate requests to the server with bearer-token authentication
Secure Domain Communication	Enable secure communication between client machines and the server using HTTPS. Configure an SSL protocol to encrypt the transmission of information and perform authentication using .pfx certificates.

User Authorization

Define User Roles	Use Roles to define the set of features that a group of users will have access to
Manage User Roles	Add new roles, delete an existing role, or edit role resource to manage access to different APIs and commands
Manage Users	Register new users, assign them to a role, activate or deactivate user accounts, and view the complete list of registered users, along with the roles assigned using a unified user management interface

Ensure Reusability of Virtual Data Models with Parameterization

Like Centerprise, Astera Data Virtualization allows users to parameterize variables to support multiple configuration or deployment scenarios. These parameters are then replaced during runtime with respective values to offer greater flexibility and reusability and virtual data model deployments.

Parameters

Config Parameters	Eliminate the use of hardcoded values and support multiple configurations by specifying parameters in the Config Parameters page
Parameter Values	String representation of the allowed value type or a variable denoted with the $ symbol surrounded by parenthesis. These are replaced with their assigned value in the Config file during runtime.
Config File	Store project-wide variables and their values in the Config file that can be associated with a particular project deployment.

*These features will be released in the future.