Support ScannedCount on DynamoDb Document Model #3751

irina-herciu · 2025-04-09T09:50:50Z

Expose ScannedCount on Amazon.DynamoDBv2.DocumentModel.Search

Description

Add a new ScannedCount property to Search class.
Populate this value internally from the most recent ScanResponse or QueryResponse.
Accumulate ScannedCount value for GetRemaining/GetRemainingAsync

Motivation and Context

#2018

Testing

Unit tests added for new property added

Screenshots (if appropriate)

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist

My code follows the code style of this project
My change requires a change to the documentation
I have updated the documentation accordingly
I have read the README document
I have added tests to cover my changes
All new and existing tests passed

License

I confirm that this pull request can be released under the Apache 2 license

normj

My comments are affect both the sync and async code paths.

sdk/src/Services/DynamoDBv2/Custom/DocumentModel/Search.cs

dscpinheiro · 2025-04-14T20:04:59Z

sdk/src/Services/DynamoDBv2/Properties/AssemblyInfo.cs

@@ -18,6 +18,7 @@
 #else
 #error Unknown platform constant - unable to set correct AssemblyDescription
 #endif
+[assembly: InternalsVisibleTo("AWSSDK.UnitTests.DynamoDBv2.NetFramework, PublicKey=0024000004800000940000000602000000240000525341310004000001000100db5f59f098d27276c7833875a6263a3cc74ab17ba9a9df0b52aedbe7252745db7274d5271fd79c1f08f668ecfa8eaab5626fa76adc811d3c8fc55859b0d09d3bc0a84eecd0ba891f2b8a2fc55141cdcc37c2053d53491e650a479967c3622762977900eddbf1252ed08a2413f00a28f3a0752a81203f03ccb7f684db373518b4")]


FYI this file is generated, it'll be overwritten by the generator the next time it runs.

I do see why you had to add it since a lot of the Search related types are internal, but we may need a different approach here (e.g. would it be better to add an integration test instead? Or revisit the access modifiers to make them public?)

@dscpinheiro in this case is there any issue if I update the AssemblyInfo template and add assembly: InternalsVisibleTo for unit tests projects which looks like respects naming convention AWSSDK.UnitTests.{AssemblyTitle}.NetFramework ?

That could work, but I think you'd also need to update the test projects to be signed... I'm honestly not a fan of making internals visible; I know we did that for Core but I wish we didn't (which is why I mentioned maybe using an integration test instead if possible).

We can until Norm's back to get his opinion.

Discussed with @dscpinheiro and I'm okay with relying on InternalsVisibleTo to make it easier to write unit tests. The generator will need to be updated to write this InternalsVisibleTo since the AssemblyInfo.cs is generated.

When doing the generator update you will need to add the logic to check if it is generating the DynamoDB AssemblyInfo and if so add the attribute. I don't think we want to add this attribute to every service's AssemblyInfo.

templates and generator cod was updated for DynamoDB service projects. @normj please have another look.

sdk/src/Services/DynamoDBv2/Custom/DocumentModel/Search.cs

normj · 2025-05-09T21:17:22Z

sdk/src/Services/DynamoDBv2/Properties/AssemblyInfo.cs

@@ -18,6 +18,7 @@
 #else
 #error Unknown platform constant - unable to set correct AssemblyDescription
 #endif
+[assembly: InternalsVisibleTo("AWSSDK.UnitTests.DynamoDBv2.NetFramework, PublicKey=0024000004800000940000000602000000240000525341310004000001000100db5f59f098d27276c7833875a6263a3cc74ab17ba9a9df0b52aedbe7252745db7274d5271fd79c1f08f668ecfa8eaab5626fa76adc811d3c8fc55859b0d09d3bc0a84eecd0ba891f2b8a2fc55141cdcc37c2053d53491e650a479967c3622762977900eddbf1252ed08a2413f00a28f3a0752a81203f03ccb7f684db373518b4")]


Discussed with @dscpinheiro and I'm okay with relying on InternalsVisibleTo to make it easier to write unit tests. The generator will need to be updated to write this InternalsVisibleTo since the AssemblyInfo.cs is generated.

When doing the generator update you will need to add the logic to check if it is generating the DynamoDB AssemblyInfo and if so add the attribute. I don't think we want to add this attribute to every service's AssemblyInfo.

normj

I left minor comment and running the PR through the internal build system. Assuming that is successful I can approve.

generator/ServiceClientGeneratorLib/UnitTestProjectFileCreator.cs

normj · 2025-05-20T16:58:24Z

Can you regenerate the Assembly.tt file so you get an updated Assembly.cs file? Looks like it got merged weird at one point in your branch and the version section is being generated without the #if of the conditional causing the build break. Once you regenenrate the Assembly.cs from the Assembly.tt the #else and #endif should go away.

In case you don't know what I mean in the Generator solution right click on the Assembly.tt file and pick Run Custom Tool

// You can specify all the values or you can default the Build and Revision Numbers 
// by using the '*' as shown below:
// [assembly: AssemblyVersion("1.0.*")]
[assembly: AssemblyVersion("4.0")]
#else
[assembly: AssemblyVersion("4.0.0.6")]
#endif
[assembly: AssemblyFileVersion("4.0.0.6")]

normj · 2025-05-21T00:10:17Z

I would also suggest avoiding commit the generated unit project file changes. That is how you will get conflicts like you have now. The generated changes will get picked and committed during the next release after this is merged.

irina-herciu · 2025-05-21T06:23:28Z

Can you regenerate the Assembly.tt file so you get an updated Assembly.cs file? Looks like it got merged weird at one point in your branch and the version section is being generated without the #if of the conditional causing the build break. Once you regenenrate the Assembly.cs from the Assembly.tt the #else and #endif should go away.

In case you don't know what I mean in the Generator solution right click on the Assembly.tt file and pick Run Custom Tool
// You can specify all the values or you can default the Build and Revision Numbers 
// by using the '*' as shown below:
// [assembly: AssemblyVersion("1.0.*")]
[assembly: AssemblyVersion("4.0")]
#else
[assembly: AssemblyVersion("4.0.0.6")]
#endif
[assembly: AssemblyFileVersion("4.0.0.6")]

Assembly.cs was regenerated and the #else and #endif is removed. @normj Can you have another look?

normj · 2025-05-21T22:46:20Z

Not sure what happen with my previous internal build run that triggered the issues with AssemblyInfo. I assume my local branch was in a bad state and deleted and repulled. And I don't see the same problem. I push the branch through the internal build system again.

You do need to address the merge conflict. That should just mean merge in the latest development branch into your branch. The unit tests for Private5G was deleted in development which is causing the merge conflict.

normj · 2025-05-22T22:53:02Z

.NET Framework build is failing with the following build errors


C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(24,27): error CS1729: 'Search' does not contain a constructor that takes 0 arguments [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
--
2747 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(26,17): error CS0117: 'Search' does not contain a definition for 'SourceTable' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2748 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(27,17): error CS0200: Property or indexer 'Search.TableName' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2749 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(28,17): error CS0200: Property or indexer 'Search.CollectResults' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2750 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(29,17): error CS0200: Property or indexer 'Search.Filter' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2751 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(41,13): error CS1929: 'Search' does not contain a definition for 'Reset' and the best extension method overload 'MockExtensions.Reset(Mock)' requires a receiver of type 'Moq.Mock' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2752 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(68,34): error CS1061: 'Search' does not contain a definition for 'GetNextSetHelper' and no accessible extension method 'GetNextSetHelper' accepting a first argument of type 'Search' could be found (are you missing a using directive or an assembly reference?) [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2753 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(110,34): error CS1061: 'Search' does not contain a definition for 'GetRemainingHelper' and no accessible extension method 'GetRemainingHelper' accepting a first argument of type 'Search' could be found (are you missing a using directive or an assembly reference?) [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2754 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(135,13): error CS1929: 'Search' does not contain a definition for 'Reset' and the best extension method overload 'MockExtensions.Reset(Mock)' requires a receiver of type 'Moq.Mock' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2755 | 0 Warning(s)
2756 | 9 Error(s)

irina-herciu · 2025-05-23T08:33:11Z

.NET Framework build is failing with the following build errors


C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(24,27): error CS1729: 'Search' does not contain a constructor that takes 0 arguments [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
--
2747 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(26,17): error CS0117: 'Search' does not contain a definition for 'SourceTable' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2748 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(27,17): error CS0200: Property or indexer 'Search.TableName' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2749 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(28,17): error CS0200: Property or indexer 'Search.CollectResults' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2750 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(29,17): error CS0200: Property or indexer 'Search.Filter' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2751 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(41,13): error CS1929: 'Search' does not contain a definition for 'Reset' and the best extension method overload 'MockExtensions.Reset(Mock)' requires a receiver of type 'Moq.Mock' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2752 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(68,34): error CS1061: 'Search' does not contain a definition for 'GetNextSetHelper' and no accessible extension method 'GetNextSetHelper' accepting a first argument of type 'Search' could be found (are you missing a using directive or an assembly reference?) [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2753 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(110,34): error CS1061: 'Search' does not contain a definition for 'GetRemainingHelper' and no accessible extension method 'GetRemainingHelper' accepting a first argument of type 'Search' could be found (are you missing a using directive or an assembly reference?) [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2754 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(135,13): error CS1929: 'Search' does not contain a definition for 'Reset' and the best extension method overload 'MockExtensions.Reset(Mock)' requires a receiver of type 'Moq.Mock' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2755 | 0 Warning(s)
2756 | 9 Error(s)

added InternalsVisibleTo for AWSSDK.UnitTests.NetFramework to fix the build

Support ScannedCount on DynamoDb Document Model

10ab70b

dscpinheiro added the v4 label Apr 9, 2025

GarrettBeatty requested review from normj and GarrettBeatty April 9, 2025 17:26

normj requested changes Apr 11, 2025

View reviewed changes

sdk/src/Services/DynamoDBv2/Custom/DocumentModel/Search.cs Show resolved Hide resolved

sdk/src/Services/DynamoDBv2/Custom/DocumentModel/Search.cs Show resolved Hide resolved

dscpinheiro reviewed Apr 14, 2025

View reviewed changes

normj requested changes May 9, 2025

View reviewed changes

irina-herciu added 2 commits May 12, 2025 13:17

update unit test proj generation for dynamoDB

95bfd64

sync with main

f790228

normj reviewed May 20, 2025

View reviewed changes

generator/ServiceClientGeneratorLib/UnitTestProjectFileCreator.cs Outdated Show resolved Hide resolved

sign all unit test projects

8e927a0

merge from main

b5767bc

add InternalsVisibleTo to AWSSDK.UnitTests.NetFramework for dynamoDB

20fc838

Support ScannedCount on DynamoDb Document Model #3751

Are you sure you want to change the base?

Support ScannedCount on DynamoDb Document Model #3751

Conversation

irina-herciu commented Apr 9, 2025

Description

Motivation and Context

Testing

Screenshots (if appropriate)

Types of changes

Checklist

License

Uh oh!

normj left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dscpinheiro Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

irina-herciu Apr 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dscpinheiro Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

normj May 9, 2025

Choose a reason for hiding this comment

Uh oh!

irina-herciu May 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

normj May 9, 2025

Choose a reason for hiding this comment

Uh oh!

normj left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

normj commented May 20, 2025

Uh oh!

normj commented May 21, 2025

Uh oh!

irina-herciu commented May 21, 2025

Uh oh!

normj commented May 21, 2025

Uh oh!

normj commented May 22, 2025

Uh oh!

irina-herciu commented May 23, 2025

Uh oh!

irina-herciu Apr 15, 2025 •

edited

Loading