Comments in Quantitative Data Export

Status: In Progress Priority: Medium Assignee: @nurikarakaya Labels: enhancement Blocks: #2002 (Plan work for adding comments to quantitative data export)

Problem Statement

Users performing systematic reviews need to export quantitative annotation data including optional comments made during the annotation process. Currently, the quantitative data export only includes the structured annotation responses, but comments that provide additional context about the annotations are not included in the export.

This limits researchers' ability to:

• Review the rationale behind specific annotations
• Include qualitative context in their analysis
• Maintain a complete record of the review process
• Understand annotator reasoning when reconciling conflicts

Proposed Solution

Add an optional setting to the quantitative data export that allows administrators to include or exclude comment fields for each annotation question.

User Flow:

1. Admin navigates to data export settings
2. Admin sees new option: "Include annotation comments in quantitative export"
3. Default behavior: Comments are NOT included (opt-in feature)
4. When enabled, export includes additional comment columns for each annotation question
5. Export format: Each question gets an additional column for comments

Technical Approach:

• Frontend (Web): Add UI toggle in export configuration
• New checkbox: "Include annotation comments"
• Default state: unchecked

• Persist setting in export configuration

• Backend (Project Management): Update data export service

• Respect configuration setting
• Pull comment data from annotations
• Add comment columns to export headers
• Include comment data in export rows

Scope:

• ✅ Include comments for annotation questions only
• ❌ Do NOT include comments for bibliographic info (no comments available)
• ✅ Comments are optional - only included if admin enables the setting

Acceptance Criteria

Planning Phase (Issue #2002)

• Step 1: Understand data export from user perspective (help guide + web app)
• Step 2: Understand functionality goal - only add comments column for annotation questions
• Step 3: Code overview - understand headers and data rows generation
• Step 4: Write implementation plan

Implementation Phase (Issue #2162)

Frontend (Web):

• Add "Include annotation comments" checkbox to export configuration UI
• Checkbox defaults to unchecked (comments excluded by default)
• Save setting in export configuration state
• Pass setting to backend export API

Backend (Project Management):

• Update export service to accept "includeComments" parameter
• Modify header generation to add comment columns when enabled
• Modify data row generation to include comment data when enabled
• Ensure comments only added for annotation questions (not bibliographic info)
• Test export with various question types

Testing:

• Unit tests for export service with/without comments
• Integration tests for frontend → backend flow
• Manual testing with real project data
• Verify export format with comments enabled/disabled
• Test edge cases (no comments, missing comments, special characters)

Documentation:

• Update user guide with new export option
• Update API documentation for export endpoint

Technical Notes

Architecture

Data Flow:

Web UI (Export Config)
  ↓ (includeComments: boolean)
Project Management API (/export)
  ↓
Data Export Service
  ↓
Quantitative Export Builder
  ↓
Excel/CSV File (with comment columns)

Implementation Details

Export Format (with comments enabled):

Study ID	Question 1	Question 1 Comments	Question 2	Question 2 Comments	...
123	Yes	Clear evidence	No	Insufficient data	...
124	No		Yes	See figure 2	...

Export Format (without comments - default):

Study ID	Question 1	Question 2	...
123	Yes	No	...
124	No	Yes	...

Database Schema

Annotations already include comments field:

• Collection: Annotations
• Field: comment (string, optional)
• Query: Join annotations with questions to build export

Related Code

Frontend (Web):

• Component: Export configuration dialog
• State: Export settings object
• API: POST /api/export with includeComments parameter

Backend (Project Management):

• Service: Data export service
• Method: GenerateQuantitativeExport
• Parameter: ExportConfiguration with IncludeComments boolean

Related Issues

• Blocks: #2002 (Planning work for this feature)
• Original Request: #1413 (Comments field should be included in quantitative data export)
• PRs in Progress:
• https://github.com/camaradesuk/syrf-projectmanagement/pull/153
• https://github.com/camaradesuk/syrf-web/pull/2126

Dependencies

• No external dependencies
• Uses existing annotation comment data
• Extends existing export functionality

Testing Strategy

Unit Tests

Backend:

[Test]
public void QuantitativeExport_WithComments_IncludesCommentColumns()
{
    // Arrange
    var config = new ExportConfiguration { IncludeComments = true };

    // Act
    var export = exportService.GenerateQuantitativeExport(projectId, config);

    // Assert
    Assert.That(export.Headers, Contains.Item("Question 1 Comments"));
}

[Test]
public void QuantitativeExport_WithoutComments_ExcludesCommentColumns()
{
    // Arrange
    var config = new ExportConfiguration { IncludeComments = false };

    // Act
    var export = exportService.GenerateQuantitativeExport(projectId, config);

    // Assert
    Assert.That(export.Headers, Does.Not.Contain("Question 1 Comments"));
}

Frontend:

it('should default to comments disabled', () => {
  const fixture = TestBed.createComponent(ExportConfigComponent);
  const component = fixture.componentInstance;

  expect(component.exportConfig.includeComments).toBe(false);
});

it('should enable comments when checkbox is checked', () => {
  const fixture = TestBed.createComponent(ExportConfigComponent);
  const checkbox = fixture.nativeElement.querySelector('#include-comments');

  checkbox.click();

  expect(component.exportConfig.includeComments).toBe(true);
});

Integration Tests

• Full export flow: UI → API → Excel file generation
• Verify Excel file contains correct columns and data
• Test with projects that have/don't have comments

Manual Testing

• Export data from real project with various annotation types
• Verify comment columns appear correctly
• Check special characters in comments (quotes, commas, newlines)
• Test with large datasets (performance)

Success Metrics

• ✅ Default behavior: Comments excluded (backward compatible)
• ✅ When enabled: Comment columns appear for all annotation questions
• ✅ No comments for bibliographic info (as expected)
• ✅ Export file size increase acceptable (<20% with comments)
• ✅ Export performance unchanged (<1 second difference)

Timeline

Started: 2025-09-08 (planning - #2002) Implementation Started: 2025-11-10 Latest Update: 2025-11-13 (PRs in progress) Target Completion: TBD

User Impact

Benefits:

• ✅ Complete data export with qualitative context
• ✅ Better understanding of annotation rationale
• ✅ Improved conflict resolution visibility
• ✅ More comprehensive research records

Risks:

• ⚠️ Larger export files (mitigated by opt-in design)
• ⚠️ Potential PII in comments (admin responsibility to review)

---

Source: GitHub Issue #2162 Last Synced: 2025-11-24 16:10

This feature brief was auto-generated from the GitHub issue. Implementation is in progress with PRs for both backend and frontend components. The feature follows the planned approach from issue #2002 and adds valuable qualitative context to quantitative data exports.

Next Actions:

1. Review PRs #153 (project-management) and #2126 (web)
2. Complete unit and integration tests
3. Manual testing with real project data
4. Update user guide documentation
5. Deploy to staging for user acceptance testing