Skip to content

Support ScannedCount on DynamoDb Document Model #3751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: development
Choose a base branch
from

Conversation

irina-herciu
Copy link
Contributor

Expose ScannedCount on Amazon.DynamoDBv2.DocumentModel.Search

Description

Add a new ScannedCount property to Search class.
Populate this value internally from the most recent ScanResponse or QueryResponse.
Accumulate ScannedCount value for GetRemaining/GetRemainingAsync

Motivation and Context

#2018

Testing

Unit tests added for new property added

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project
  • My change requires a change to the documentation
  • I have updated the documentation accordingly
  • I have read the README document
  • I have added tests to cover my changes
  • All new and existing tests passed

License

  • I confirm that this pull request can be released under the Apache 2 license

Sorry, something went wrong.

Copy link
Member

@normj normj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My comments are affect both the sync and async code paths.

@@ -18,6 +18,7 @@
#else
#error Unknown platform constant - unable to set correct AssemblyDescription
#endif
[assembly: InternalsVisibleTo("AWSSDK.UnitTests.DynamoDBv2.NetFramework, PublicKey=0024000004800000940000000602000000240000525341310004000001000100db5f59f098d27276c7833875a6263a3cc74ab17ba9a9df0b52aedbe7252745db7274d5271fd79c1f08f668ecfa8eaab5626fa76adc811d3c8fc55859b0d09d3bc0a84eecd0ba891f2b8a2fc55141cdcc37c2053d53491e650a479967c3622762977900eddbf1252ed08a2413f00a28f3a0752a81203f03ccb7f684db373518b4")]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI this file is generated, it'll be overwritten by the generator the next time it runs.

I do see why you had to add it since a lot of the Search related types are internal, but we may need a different approach here (e.g. would it be better to add an integration test instead? Or revisit the access modifiers to make them public?)

Copy link
Contributor Author

@irina-herciu irina-herciu Apr 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dscpinheiro in this case is there any issue if I update the AssemblyInfo template and add assembly: InternalsVisibleTo for unit tests projects which looks like respects naming convention AWSSDK.UnitTests.{AssemblyTitle}.NetFramework ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could work, but I think you'd also need to update the test projects to be signed... I'm honestly not a fan of making internals visible; I know we did that for Core but I wish we didn't (which is why I mentioned maybe using an integration test instead if possible).

We can until Norm's back to get his opinion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed with @dscpinheiro and I'm okay with relying on InternalsVisibleTo to make it easier to write unit tests. The generator will need to be updated to write this InternalsVisibleTo since the AssemblyInfo.cs is generated.

When doing the generator update you will need to add the logic to check if it is generating the DynamoDB AssemblyInfo and if so add the attribute. I don't think we want to add this attribute to every service's AssemblyInfo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

templates and generator cod was updated for DynamoDB service projects. @normj please have another look.

@@ -18,6 +18,7 @@
#else
#error Unknown platform constant - unable to set correct AssemblyDescription
#endif
[assembly: InternalsVisibleTo("AWSSDK.UnitTests.DynamoDBv2.NetFramework, PublicKey=0024000004800000940000000602000000240000525341310004000001000100db5f59f098d27276c7833875a6263a3cc74ab17ba9a9df0b52aedbe7252745db7274d5271fd79c1f08f668ecfa8eaab5626fa76adc811d3c8fc55859b0d09d3bc0a84eecd0ba891f2b8a2fc55141cdcc37c2053d53491e650a479967c3622762977900eddbf1252ed08a2413f00a28f3a0752a81203f03ccb7f684db373518b4")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed with @dscpinheiro and I'm okay with relying on InternalsVisibleTo to make it easier to write unit tests. The generator will need to be updated to write this InternalsVisibleTo since the AssemblyInfo.cs is generated.

When doing the generator update you will need to add the logic to check if it is generating the DynamoDB AssemblyInfo and if so add the attribute. I don't think we want to add this attribute to every service's AssemblyInfo.

Copy link
Member

@normj normj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left minor comment and running the PR through the internal build system. Assuming that is successful I can approve.

@normj
Copy link
Member

normj commented May 20, 2025

Can you regenerate the Assembly.tt file so you get an updated Assembly.cs file? Looks like it got merged weird at one point in your branch and the version section is being generated without the #if of the conditional causing the build break. Once you regenenrate the Assembly.cs from the Assembly.tt the #else and #endif should go away.

In case you don't know what I mean in the Generator solution right click on the Assembly.tt file and pick Run Custom Tool

// You can specify all the values or you can default the Build and Revision Numbers 
// by using the '*' as shown below:
// [assembly: AssemblyVersion("1.0.*")]
[assembly: AssemblyVersion("4.0")]
#else
[assembly: AssemblyVersion("4.0.0.6")]
#endif
[assembly: AssemblyFileVersion("4.0.0.6")]

@normj
Copy link
Member

normj commented May 21, 2025

I would also suggest avoiding commit the generated unit project file changes. That is how you will get conflicts like you have now. The generated changes will get picked and committed during the next release after this is merged.

@irina-herciu
Copy link
Contributor Author

Can you regenerate the Assembly.tt file so you get an updated Assembly.cs file? Looks like it got merged weird at one point in your branch and the version section is being generated without the #if of the conditional causing the build break. Once you regenenrate the Assembly.cs from the Assembly.tt the #else and #endif should go away.

In case you don't know what I mean in the Generator solution right click on the Assembly.tt file and pick Run Custom Tool

// You can specify all the values or you can default the Build and Revision Numbers 
// by using the '*' as shown below:
// [assembly: AssemblyVersion("1.0.*")]
[assembly: AssemblyVersion("4.0")]
#else
[assembly: AssemblyVersion("4.0.0.6")]
#endif
[assembly: AssemblyFileVersion("4.0.0.6")]

Assembly.cs was regenerated and the #else and #endif is removed. @normj Can you have another look?

@normj
Copy link
Member

normj commented May 21, 2025

Not sure what happen with my previous internal build run that triggered the issues with AssemblyInfo. I assume my local branch was in a bad state and deleted and repulled. And I don't see the same problem. I push the branch through the internal build system again.

You do need to address the merge conflict. That should just mean merge in the latest development branch into your branch. The unit tests for Private5G was deleted in development which is causing the merge conflict.

@normj
Copy link
Member

normj commented May 22, 2025

.NET Framework build is failing with the following build errors


C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(24,27): error CS1729: 'Search' does not contain a constructor that takes 0 arguments [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
--
2747 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(26,17): error CS0117: 'Search' does not contain a definition for 'SourceTable' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2748 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(27,17): error CS0200: Property or indexer 'Search.TableName' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2749 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(28,17): error CS0200: Property or indexer 'Search.CollectResults' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2750 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(29,17): error CS0200: Property or indexer 'Search.Filter' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2751 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(41,13): error CS1929: 'Search' does not contain a definition for 'Reset' and the best extension method overload 'MockExtensions.Reset(Mock)' requires a receiver of type 'Moq.Mock' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2752 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(68,34): error CS1061: 'Search' does not contain a definition for 'GetNextSetHelper' and no accessible extension method 'GetNextSetHelper' accepting a first argument of type 'Search' could be found (are you missing a using directive or an assembly reference?) [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2753 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(110,34): error CS1061: 'Search' does not contain a definition for 'GetRemainingHelper' and no accessible extension method 'GetRemainingHelper' accepting a first argument of type 'Search' could be found (are you missing a using directive or an assembly reference?) [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2754 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(135,13): error CS1929: 'Search' does not contain a definition for 'Reset' and the best extension method overload 'MockExtensions.Reset(Mock)' requires a receiver of type 'Moq.Mock' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2755 | 0 Warning(s)
2756 | 9 Error(s)

@irina-herciu
Copy link
Contributor Author

.NET Framework build is failing with the following build errors


C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(24,27): error CS1729: 'Search' does not contain a constructor that takes 0 arguments [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
--
2747 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(26,17): error CS0117: 'Search' does not contain a definition for 'SourceTable' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2748 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(27,17): error CS0200: Property or indexer 'Search.TableName' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2749 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(28,17): error CS0200: Property or indexer 'Search.CollectResults' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2750 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(29,17): error CS0200: Property or indexer 'Search.Filter' cannot be assigned to -- it is read only [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2751 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(41,13): error CS1929: 'Search' does not contain a definition for 'Reset' and the best extension method overload 'MockExtensions.Reset(Mock)' requires a receiver of type 'Moq.Mock' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2752 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(68,34): error CS1061: 'Search' does not contain a definition for 'GetNextSetHelper' and no accessible extension method 'GetNextSetHelper' accepting a first argument of type 'Search' could be found (are you missing a using directive or an assembly reference?) [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2753 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(110,34): error CS1061: 'Search' does not contain a definition for 'GetRemainingHelper' and no accessible extension method 'GetRemainingHelper' accepting a first argument of type 'Search' could be found (are you missing a using directive or an assembly reference?) [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2754 | C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\Services\DynamoDBv2\UnitTests\Custom\SearchTests.cs(135,13): error CS1929: 'Search' does not contain a definition for 'Reset' and the best extension method overload 'MockExtensions.Reset(Mock)' requires a receiver of type 'Moq.Mock' [C:\codebuild\tmp\output\src1527145343\src\aws-sdk-net\sdk\test\UnitTests\AWSSDK.UnitTests.NetFramework.csproj]
2755 | 0 Warning(s)
2756 | 9 Error(s)

added InternalsVisibleTo for AWSSDK.UnitTests.NetFramework to fix the build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants