[update] enhanced ROADMAP.md with advanced geographic tagging, hierarchical filtering, migration strategies, and expanded analytics features, clarified project phase details, and incorporated tagging upgrades into future plans

README.md

@@ -6,11 +6,14 @@ Owly News Summariser is a modern web application that fetches news articles from
 
 - Fetches news from configurable RSS feeds
 - Automatically summarizes articles using Ollama LLM
-- Filters news by country
+- **AI-powered intelligent tagging** with geographic, category, and source tags
+- **Advanced multi-criteria filtering** with hierarchical tag support
 - Progressive Web App (PWA) support for offline access
 - Scheduled background updates
 - High-performance Rust backend for optimal resource usage
 - Modern Vue.js frontend with TypeScript support
+- **Comprehensive analytics** and reading statistics
+- **Flexible sharing system** with multiple format options
 
 ## Project Structure
 
@@ -24,7 +27,7 @@ The project consists of multiple components:
 
 ### For Rust Backend (Recommended)
 - Rust 1.88.0+
-- [Ollama](https://ollama.ai/) for article summarization
+- [Ollama](https://ollama.ai/) for article summarization and tagging
 - SQLite (handled automatically by SQLx)
 
 ### For Python Backend (Legacy)
@@ -102,6 +105,32 @@ The project consists of multiple components:
 
    The frontend will be available at http://localhost:5173
 
+## Key Features
+
+### Intelligent Content Organization
+- **AI-Powered Tagging**: Automatic classification with geographic, topical, and source tags
+- **Hierarchical Filtering**: Multi-level filtering by location (country → region → city), categories, and content types
+- **Smart Search**: Advanced filtering with suggestions based on tag relationships and usage patterns
+- **Legacy Migration**: Seamless upgrade from simple country-based filtering to comprehensive tag-based system
+
+### Advanced Analytics
+- **Reading Statistics**: Track reading time, completion rates, and engagement patterns
+- **Content Analytics**: Source performance, tag usage, and trending topics analysis
+- **Geographic Insights**: Location-based content distribution and reading preferences
+- **Goal Tracking**: Personal reading goals with progress monitoring
+
+### Flexible Article Display
+- **Compact View**: Title, excerpt, tags, and action buttons for quick browsing
+- **On-Demand Loading**: Full content, AI summaries, and source links as needed
+- **Visual Tag System**: Color-coded, hierarchical tags with click-to-filter functionality
+- **Reading Status**: Visual indicators for read/unread status and progress tracking
+
+### Enhanced Sharing
+- **Multiple Formats**: Text, Markdown, HTML, and JSON export options
+- **Custom Templates**: User-configurable sharing formats
+- **One-Click Operations**: Copy to clipboard with formatted content
+- **Privacy Controls**: Configurable information inclusion in shared content
+
 ## Building for Production
 
 ### Building the Rust Backend
@@ -156,3 +185,33 @@ Run frontend tests:
 cd frontend
 npm run test
 ```
+
+## Configuration
+
+The application uses a comprehensive configuration system via `config.toml`:
+
+- **AI Settings**: Configure Ollama integration for summaries and tagging
+- **Display Preferences**: Default views, themes, and UI customization
+- **Analytics**: Control data collection and retention policies
+- **Filtering**: Smart suggestions, saved filters, and geographic hierarchy
+- **Sharing**: Default formats and custom templates
+
+See the example configuration in the project for detailed options.
+
+## Migration from Legacy Systems
+
+The application includes automatic migration tools for upgrading from simpler filtering systems:
+
+- **Country Filter Migration**: Automatic conversion to hierarchical geographic tags
+- **Data Preservation**: Maintains historical data during migration
+- **Backward Compatibility**: Gradual transition with user control
+- **Validation Tools**: Ensure data integrity throughout the migration process
+
+## Future Roadmap
+
+The project is evolving through three phases:
+1. **Phase 1**: High-performance Rust backend with advanced filtering and analytics
+2. **Phase 2**: CLI application for power users and automation
+3. **Phase 3**: Migration to Dioxus for a full Rust stack
+
+See `ROADMAP.md` for detailed development plans and architectural decisions.

backend-rust/ROADMAP.md

@@ -6,6 +6,7 @@ This document outlines the strategic approach for transforming the project throu
 
 ### Current Phase: Axum API Setup
 ```
+
 owly-news-summariser/
 ├── src/
 │   ├── main.rs              # Entry point (will evolve)
@@ -39,6 +40,7 @@ owly-news-summariser/
 ```
 ### Phase 2: Multi-Binary Structure (API + CLI)
 ```
+
 owly-news-summariser/
 ├── src/
 │   ├── lib.rs               # Shared library code
@@ -57,6 +59,7 @@ owly-news-summariser/
 ```
 ### Phase 3: Full Rust Stack
 ```
+
 owly-news-summariser/
 ├── src/
 │   ├── [same structure as Phase 2]
@@ -67,7 +70,6 @@ owly-news-summariser/
 ├── config.toml
 └── Cargo.toml
 ```
-
 ## Core Features & Architecture
 
 ### Article Processing & Display Workflow
@@ -96,11 +98,13 @@ owly-news-summariser/
 
 3. **Intelligent Tagging System**
     - **Automatic Tag Generation**: AI analyzes content and assigns relevant tags
-    - **Visual Tag Display**: Color-coded tags in compact article view
+    - **Geographic & Source Tags**: AI-generated location tags (countries, regions, cities) and publication source tags
-    - **Tag Categories**: Technology, Politics, Business, Sports, Health, etc.
+    - **Content Category Tags**: Technology, Politics, Business, Sports, Health, etc.
-    - **Tag Filtering**: Quick filtering by clicking tags
+    - **Visual Tag Display**: Color-coded tags in compact article view with hierarchical display
+    - **Tag Filtering**: Quick filtering by clicking tags with smart suggestions
     - **Custom Tags**: User-defined tags and categories
     - **Tag Confidence**: Visual indicators for AI vs manual tags
+    - **Tag Migration**: Automatic conversion of legacy country filters to geographic tags
 
 4. **Analytics & Statistics System**
     - **Reading Analytics**:
@@ -121,19 +125,23 @@ owly-news-summariser/
 5. **Advanced Filtering System**
     - **Multi-Criteria Filtering**:
         - By tags (single or multiple with AND/OR logic)
+        - By geographic tags (country, region, city with hierarchical filtering)
+        - By content categories and topics
         - By date ranges (posted, added, read)
         - By processing status (pending, completed, failed)
         - By content availability (scraped, summary, RSS-only)
         - By read/unread status
+    - **Smart Filter Migration**: Automatic conversion of legacy country filters to tag-based equivalents
     - **Saved Filter Presets**:
         - Custom filter combinations
         - Quick access to frequent searches
-    - **Smart Suggestions**: Filter suggestions based on usage patterns
+        - Geographic preset templates (e.g., "European Tech News", "US Politics")
+    - **Smart Suggestions**: Filter suggestions based on usage patterns and tag relationships
 
 6. **Settings & Management System**
     - **User Preferences**:
         - Default article view mode
-        - Tag display preferences
+        - Tag display preferences with geographic hierarchy settings
         - Reading tracking settings
         - Notification preferences
     - **System Settings**:
@@ -145,6 +153,7 @@ owly-news-summariser/
         - Bulk operations (mark read, delete, retag)
         - Archive old articles
         - Export/import functionality
+        - Legacy data migration tools
 
 7. **Article Sharing System**
     - **Multiple Share Formats**:
@@ -157,17 +166,18 @@ owly-news-summariser/
     - **Privacy Controls**: Control what information is included in shares
 
 8. **Database Schema**
-   ```
+```
-   Articles: id, title, url, source_type, rss_content, full_content, 
+Articles: id, title, url, source_type, rss_content, full_content,
-            summary, processing_status, published_at, added_at, read_at,
+summary, processing_status, published_at, added_at, read_at,
-            read_count, reading_time, ai_enabled, created_at, updated_at
+read_count, reading_time, ai_enabled, created_at, updated_at
-   Tags: id, name, category, description, color, usage_count, created_at
+Tags: id, name, category, description, color, usage_count, parent_id, created_at
-   ArticleTags: article_id, tag_id, confidence_score, ai_generated, created_at
+ArticleTags: article_id, tag_id, confidence_score, ai_generated, created_at
-   ReadingStats: user_id, article_id, read_at, reading_time, completion_rate
+ReadingStats: user_id, article_id, read_at, reading_time, completion_rate
-   FilterPresets: id, name, filter_criteria, user_id, created_at
+FilterPresets: id, name, filter_criteria, user_id, created_at
-   Settings: key, value, category, user_id, updated_at
+Settings: key, value, category, user_id, updated_at
-   ShareTemplates: id, name, format, template_content, created_at
+ShareTemplates: id, name, format, template_content, created_at
-   ```
+LegacyMigration: old_filter_type, old_value, new_tag_ids, migrated_at
+```
 
 ## Step-by-Step Process
 
@@ -182,12 +192,15 @@ owly-news-summariser/
     - Analytics and tracking preferences
     - Sharing templates and formats
     - Filter and search settings
+    - Geographic tagging preferences
 - Establish error handling patterns with `anyhow`
 - Set up logging and analytics infrastructure
 
 **Step 2: Data Layer**
 - Design comprehensive database schema with analytics and settings support
 - Create SQLx migrations for all tables including analytics and user preferences
+- Implement hierarchical tag system with geographic and content categories
+- Add legacy migration support for country filters
 - Implement article models with reading tracking and statistics
 - Add settings and preferences data layer
 - Create analytics data models and aggregation queries
@@ -199,6 +212,11 @@ owly-news-summariser/
     - RSS feed fetching and parsing
     - Web scraping with quality tracking
     - AI services for summary and tagging
+- **Enhanced Tagging Service**:
+    - Geographic location detection and tagging
+    - Content category classification
+    - Hierarchical tag relationships
+    - Legacy filter migration logic
 - **Analytics Service**:
     - Reading statistics collection and aggregation
     - Content performance metrics
@@ -213,9 +231,10 @@ owly-news-summariser/
     - Template processing
     - Privacy-aware content filtering
 - **Advanced Filtering Service**:
-    - Complex query building
+    - Complex query building with geographic hierarchy
     - Filter preset management
     - Search optimization
+    - Legacy filter migration
 
 **Step 4: Comprehensive API Layer**
 - **Article Management Routes**:
@@ -232,20 +251,23 @@ owly-news-summariser/
     - `GET /api/analytics/content-stats` - Content and source analytics
     - `GET /api/analytics/trends` - Trending topics and patterns
     - `GET /api/analytics/export` - Export analytics data
-- **Filtering & Search Routes**:
+- **Enhanced Filtering & Search Routes**:
     - `GET /api/filters/presets` - Get saved filter presets
     - `POST /api/filters/presets` - Save new filter preset
     - `GET /api/search/suggestions` - Get search and filter suggestions
     - `POST /api/search` - Advanced search with multiple criteria
+    - `POST /api/filters/migrate` - Migrate legacy country filters to tags
 - **Settings Routes**:
     - `GET /api/settings` - Get all user settings
     - `PUT /api/settings` - Update user settings
     - `GET /api/settings/system` - Get system configuration
     - `PUT /api/settings/system` - Update system settings (admin)
-- **Tag Management Routes**:
+- **Enhanced Tag Management Routes**:
-    - `GET /api/tags` - List tags with usage statistics
+    - `GET /api/tags` - List tags with usage statistics and hierarchy
+    - `GET /api/tags/geographic` - Get geographic tag hierarchy
     - `GET /api/tags/trending` - Get trending tags
     - `POST /api/tags/:id/follow` - Follow/unfollow tag for notifications
+    - `GET /api/tags/categories` - Get tag categories and relationships
 - **Sharing Routes**:
     - `GET /api/share/templates` - Get sharing templates
     - `POST /api/share/templates` - Create custom sharing template
@@ -255,18 +277,19 @@ owly-news-summariser/
 - **Compact Article Display**:
     - Card-based layout with title, RSS excerpt, tags, and timestamps
     - Action buttons for Full Article, Summary, and Source
-    - Visual tag indicators with click-to-filter
+    - Hierarchical tag display with geographic and category indicators
     - Reading status and progress indicators
 - **Advanced Analytics Dashboard**:
     - Reading statistics with charts and trends
     - Content source performance metrics
-    - Tag usage and trending topics
+    - Tag usage and trending topics with geographic breakdowns
     - Personal reading insights and goals
 - **Comprehensive Filtering Interface**:
-    - Multi-criteria filter builder
+    - Multi-criteria filter builder with geographic hierarchy
     - Saved filter presets with quick access
-    - Smart filter suggestions
+    - Smart filter suggestions based on tag relationships
     - Visual filter indicators and clear actions
+    - Legacy filter migration interface
 - **Settings Management Panel**:
     - User preference configuration
     - AI and processing settings
@@ -281,10 +304,11 @@ owly-news-summariser/
 **Step 6: Integration & Testing**
 - Test all API endpoints with comprehensive coverage
 - Test analytics collection and aggregation
-- Test filtering and search functionality
+- Test enhanced filtering and search functionality
+- Test legacy filter migration
 - Validate settings persistence and real-time updates
 - Test sharing functionality across different formats
-- Performance testing with large datasets
+- Performance testing with large datasets and hierarchical tags
 - Deploy and monitor
 
 ### Phase 2: CLI Application Addition
@@ -303,22 +327,27 @@ owly-news-summariser/
     - `owly open <id>` - Open source URL in browser
 - **Analytics Commands**:
     - `owly stats [--period day|week|month|year]` - Show reading statistics
-    - `owly trends [--tags|--sources|--topics]` - Display trending content
+    - `owly trends [--tags|--sources|--topics|--geo]` - Display trending content
     - `owly analytics export [--format csv|json]` - Export analytics data
 - **Management Commands**:
     - `owly settings [--get key] [--set key=value]` - Manage settings
     - `owly filters [--list|--save name|--load name]` - Manage filter presets
     - `owly cleanup [--old|--unread|--failed]` - Clean up articles
+    - `owly migrate [--from-country-filters]` - Migrate legacy data
+- **Enhanced Filtering Commands**:
+    - `owly filter [--tag] [--geo] [--category]` - Advanced filtering with geographic support
+    - `owly tags [--list|--hierarchy|--geo]` - Tag management with geographic display
 - **Sharing Commands**:
     - `owly share <id> [--format text|markdown|html]` - Generate share content
     - `owly export <id> [--template name] [--output file]` - Export article
 
 **Step 3: Advanced CLI Features**
-- Interactive filtering and search
+- Interactive filtering and search with geographic hierarchy
 - Real-time analytics display with charts (using ASCII graphs)
 - Bulk operations with progress indicators
 - Settings management with validation
 - Shell completion for all commands and parameters
+- Legacy data migration tools
 
 ### Phase 3: Dioxus Frontend Migration
 
@@ -327,16 +356,20 @@ owly-news-summariser/
     - `ArticleCard` - Compact article display with action buttons
     - `ArticleViewer` - Full article content display
     - `SummaryViewer` - AI summary display
-    - `TagCloud` - Interactive tag display and filtering
+    - `TagCloud` - Interactive tag display with geographic hierarchy
+    - `GeographicMap` - Visual geographic filtering interface
 - **Analytics Components**:
     - `AnalyticsDashboard` - Main analytics overview
     - `ReadingStats` - Personal reading statistics
     - `TrendChart` - Trending topics and patterns
     - `ContentMetrics` - Source and content analytics
-- **Filtering Components**:
+    - `GeographicAnalytics` - Location-based content insights
-    - `FilterBuilder` - Advanced filter creation interface
+- **Enhanced Filtering Components**:
+    - `FilterBuilder` - Advanced filter creation interface with geographic support
     - `FilterPresets` - Saved filter management
     - `SearchBar` - Smart search with suggestions
+    - `GeographicFilter` - Hierarchical location filtering
+    - `MigrationTool` - Legacy filter migration interface
 - **Settings Components**:
     - `SettingsPanel` - User preference management
     - `SystemConfig` - System-wide configuration
@@ -352,14 +385,14 @@ owly-news-summariser/
     - Quick preview on hover
     - Keyboard navigation support
 - **Advanced Analytics**:
-    - Interactive charts and graphs
+    - Interactive charts and graphs with geographic data
     - Customizable dashboard widgets
     - Goal setting and progress tracking
     - Comparison and trend analysis
 - **Intelligent Filtering**:
-    - Auto-complete for filters
+    - Auto-complete for filters with geographic suggestions
-    - Visual filter builder
+    - Visual filter builder with map integration
-    - Filter combination suggestions
+    - Filter combination suggestions based on tag relationships
     - Saved search notifications
 - **Seamless Sharing**:
     - One-click sharing with clipboard integration
@@ -371,8 +404,8 @@ owly-news-summariser/
 
 ### 1. Performance & Scalability
 - **Efficient Data Loading**: Lazy loading and pagination for large datasets
-- **Optimized Queries**: Indexed database queries for filtering and analytics
+- **Optimized Queries**: Indexed database queries for filtering and analytics with hierarchical tag support
-- **Caching Strategy**: Smart caching for frequently accessed content
+- **Caching Strategy**: Smart caching for frequently accessed content and tag hierarchies
 - **Real-time Updates**: WebSocket integration for live analytics
 
 ### 2. User Experience Focus
@@ -380,18 +413,20 @@ owly-news-summariser/
 - **Responsive Design**: Optimized for mobile and desktop
 - **Accessibility**: Full keyboard navigation and screen reader support
 - **Customization**: User-configurable interface and behavior
+- **Smooth Migration**: Seamless transition from country-based to tag-based filtering
 
 ### 3. Analytics & Insights
 - **Privacy-First**: User control over data collection and retention
 - **Actionable Insights**: Meaningful statistics that guide reading habits
 - **Performance Metrics**: System health and efficiency tracking
-- **Trend Analysis**: Pattern recognition for content and behavior
+- **Trend Analysis**: Pattern recognition for content and behavior with geographic context
 
 ### 4. Content Management
 - **Flexible Display**: Multiple view modes for different use cases
-- **Smart Organization**: AI-assisted content categorization
+- **Smart Organization**: AI-assisted content categorization with geographic awareness
 - **Bulk Operations**: Efficient management of large article collections
 - **Data Integrity**: Reliable content processing and error handling
+- **Legacy Support**: Smooth migration from existing country-based filtering
 
 ## Enhanced Configuration File Structure
 
@@ -420,6 +455,8 @@ enable_smart_suggestions = true
 max_recent_filters = 10
 auto_save_filters = true
 default_sort = "added_desc"  # added_desc, published_desc, title_asc
+enable_geographic_hierarchy = true
+auto_migrate_country_filters = true
 
 [sharing]
 default_format = "text"
@@ -435,6 +472,7 @@ format = """
 {summary}
 
 🏷️ Tags: {tags}
+🌍 Location: {geographic_tags}
 🔗 Source: {url}
 📅 Published: {published_at}
 
@@ -448,6 +486,7 @@ format = """
 {summary}
 
 **Tags:** {tags}  
+**Location:** {geographic_tags}  
 **Source:** [{url}]({url})  
 **Published:** {published_at}
 
@@ -471,6 +510,9 @@ temperature = 0.3
 max_tokens = 200
 max_tags_per_article = 10
 min_confidence_threshold = 0.7
+enable_geographic_tagging = true
+enable_category_tagging = true
+geographic_hierarchy_levels = 3  # country, region, city
 
 [scraping]
 timeout_seconds = 30
@@ -486,35 +528,57 @@ retry_attempts = 3
 priority_manual = true
 auto_mark_read_on_view = false
 
+[migration]
+auto_convert_country_filters = true
+preserve_legacy_data = true
+migration_batch_size = 100
+
 [cli]
 default_output = "table"
 pager_command = "less"
 show_progress = true
 auto_confirm_bulk = false
+show_geographic_hierarchy = true
 ```
 
+## Migration Strategy for Country-Based Filtering
+
+### Automatic Migration Process
+1. **Data Analysis**: Scan existing country filter data and RSS feed origins
+2. **Tag Generation**: Create geographic tags for each country with hierarchical structure
+3. **Filter Conversion**: Convert country-based filters to tag-based equivalents
+4. **User Notification**: Inform users about the migration and new capabilities
+5. **Gradual Rollout**: Maintain backward compatibility during transition period
+
+### Enhanced Geographic Features
+- **Hierarchical Display**: Country → Region → City tag hierarchy
+- **Visual Map Integration**: Interactive geographic filtering via map interface
+- **Smart Suggestions**: Related location and content suggestions
+- **Multi-Level Filtering**: Filter by specific cities, regions, or broader geographic areas
+- **Source Intelligence**: AI detection of article geographic relevance beyond RSS origin
+
 ## Future Enhancements (Post Phase 3)
 
 ### Advanced Analytics
-- **Machine Learning Insights**: Content recommendation based on reading patterns
+- **Machine Learning Insights**: Content recommendation based on reading patterns and geographic preferences
-- **Predictive Analytics**: Trending topic prediction
+- **Predictive Analytics**: Trending topic prediction with geographic context
 - **Behavioral Analysis**: Reading habit optimization suggestions
-- **Comparative Analytics**: Benchmark against reading goals and averages
+- **Comparative Analytics**: Benchmark against reading goals and regional averages
 
 ### Enhanced Content Management
-- **Smart Collections**: AI-curated article collections
+- **Smart Collections**: AI-curated article collections with geographic themes
 - **Reading Lists**: Planned reading with progress tracking
-- **Content Relationships**: Related article suggestions
+- **Content Relationships**: Related article suggestions with geographic relevance
-- **Advanced Search**: Full-text search with relevance scoring
+- **Advanced Search**: Full-text search with relevance scoring and geographic weighting
 
 ### Social & Collaboration Features
-- **Reading Groups**: Shared reading lists and discussions
+- **Reading Groups**: Shared reading lists and discussions with geographic focus
 - **Social Sharing**: Integration with social platforms
 - **Collaborative Tagging**: Community-driven content organization
-- **Reading Challenges**: Gamification of reading habits
+- **Reading Challenges**: Gamification of reading habits with geographic themes
 
 ### Integration & Extensibility
 - **Browser Extension**: Seamless article saving and reading
-- **Mobile Apps**: Native iOS/Android applications
+- **Mobile Apps**: Native iOS/Android applications with location awareness
 - **API Ecosystem**: Third-party integrations and plugins
-- **Webhook System**: Real-time notifications and integrations
+- **Webhook System**: Real-time notifications and integrations with geographic filtering