|
1 | 1 |
|
2 | | -TBD |
| 2 | +# Sample using DatabaseLoader for training an ML model directly against data in a SQL Server database (Or any relational database) |
| 3 | + |
| 4 | + |
| 5 | + |
| 6 | +| ML.NET version | API type | Status | App Type | Data type | Scenario | ML Task | Algorithms | |
| 7 | +|----------------|-------------------|-------------------------------|-------------|-----------|---------------------|---------------------------|-----------------------------| |
| 8 | +| v0.16-Preview | Dynamic API | up-to-date | Console app | SQL Server database or any relational database | IDataView from DB | Any | Any | |
| 9 | + |
| 10 | +This sample shows you how you can use the native database loader ro directly train an ML model against relational databases. This loader supports any relational database provider supported by System.Data in .NET Core or .NET Framework, meaning that you can use any RDBMS such as SQL Server, Azure SQL Database, Oracle, SQLite, PostgreSQL, MySQL, Progress, IBM DB2, etc. |
| 11 | + |
| 12 | +## Problem |
| 13 | + |
| 14 | +In the enterprise and many organizations in general, data is organized and stored as relational databases to be used by enterprise applications. Many of those organizations also prepare their ML model training/evaluation data in relational databases which is also where the new data is being collected and prepared. Therefore, many of those users would also like to directly train/evaluate ML models directly agaist that data stored in relational databases. |
| 15 | + |
| 16 | +## Background |
| 17 | + |
| 18 | +In previous [ML.NET](https://dot.net/ml) releases, since [ML.NET](https://dot.net/ml) 1.0, you could also train against a relational database by providing data through an IEnumerable collection by using the [LoadFromEnumerable()](https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.dataoperationscatalog.loadfromenumerable?view=ml-dotnet) API where the data could be coming from a relational database or any other source. However, when using that approach, you as a developer are responsible for the code reading from the relational database (such as using Entity Framework or any other approach) which needs to be implemented properly so you are streaming data while training the ML model, as in this [previous sample using LoadFromEnumerable()](https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/getting-started/DatabaseIntegration). |
| 19 | + |
| 20 | +## Solution |
| 21 | + |
| 22 | +This new Database Loader provides a much simpler code implementation for you since the way it reads from the database and makes data available through the IDataView is provided out-of-the-box by the [ML.NET](https://dot.net/ml) framework so you just need to specify your database connection string, what’s the SQL statement for the dataset columns and what’s the data-class to use when loading the data. It is that simple! |
| 23 | + |
| 24 | +Here’s example code on how easily you can now configure your code to load data directly from a relational database into an IDataView which will be used later on when training your model. |
| 25 | + |
| 26 | +```cs --source-file ./DatabaseLoaderConsoleApp/Program.cs --project ./SentimentAnalysis/SentimentAnalysisConsoleApp/SentimentAnalysisConsoleApp.csproj --editable false --region step1to3 |
| 27 | + |
| 28 | +var mlContext = new MLContext(); |
| 29 | + |
| 30 | +// The following is a connection string using a localdb SQL database, |
| 31 | +// but you can also use connection strings against on-premises SQL Server, Azure SQL Database |
| 32 | +// or any other relational database (Oracle, SQLite, PostgreSQL, MySQL, Progress, IBM DB2, etc.) |
| 33 | +
|
| 34 | +// localdb SQL database connection string using a filepath to attach the database file into localdb |
| 35 | +string dbFilePath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "SqlLocalDb", "Criteo-100k-rows.mdf"); |
| 36 | +string connectionString = $"Data Source = (LocalDB)\\MSSQLLocalDB;AttachDbFilename={dbFilePath};Database=Criteo-100k-rows;Integrated Security = True"; |
| 37 | + |
| 38 | +string commandText = "SELECT * from URLClicks"; |
| 39 | + |
| 40 | +DatabaseLoader loader = mlContext.Data.CreateDatabaseLoader<UrlClick>(); |
| 41 | + |
| 42 | +DatabaseSource dbSource = new DatabaseSource(SqlClientFactory.Instance, |
| 43 | + connectionString, |
| 44 | + commandText); |
| 45 | + |
| 46 | +IDataView dataView = loader.Load(dbSource); |
| 47 | + |
| 48 | +// From this point you can use the IDataView for training and validating an ML.NET model as in any other sample |
| 49 | +``` |
| 50 | + |
| 51 | +Check the rest of the sample training and evaluating an ML.NET model in the **program.cs** file. |
| 52 | + |
0 commit comments