Cloud Storage Module Documentation

The Cloud Storage module allows users to configure, manage, and synchronize their files between a local server and cloud storage solutions. This module is especially useful for ensuring secure storage, and enabling file access across multiple servers.

Key Features

• Cloud Integration: Seamlessly integrates with popular cloud storage providers such as Amazon S3 and DigitalOcean Spaces, ensuring compatibility with widely-used platforms.
• File Synchronization: Enables smooth synchronization of files between the local server and cloud storage.
• Third-Party Module Support: Designed to support third-party and custom modules for synchronization or direct file uploads, enhancing flexibility and extensibility.
• Multitenancy Awareness: Tailored for SaaS environments, with features to restrict tenant access to specific options or functionalities.
• Local Copy Management: Provides control over whether to retain a local copy of uploaded files for redundancy or offline access.
• Diagnostic Tools: Includes built-in comprehensive testing tools to verify connectivity and ensure correct configuration settings.
• Documentation Access: Offers direct links to setup guides and relevant resources within the module for easy reference.

Settings

The module supports all S3-compatible storage services, making it highly versatile. While thoroughly tested with AWS S3 and DigitalOcean Spaces, it also works with other modern cloud storage solutions that implement the S3 API. For optimal results, it is recommended to use storage providers that offer versioning to ensure data safety and easy retrieval of previous file versions.

Support us by using our referral code for DigitalOcean—it’s at no extra cost to you! Think of it as a small contribution to help us keep going.

• Key: The access key provided by your cloud storage service. This key is necessary for authenticating API requests.
• Secret: The secret key associated with your cloud storage account, used alongside the access key for secure authentication.
• Bucket: The name of your cloud storage bucket where files will be stored.
  Important: Bucket names must be globally unique and follow naming conventions (e.g., no spaces or uppercase letters). A helpful link to bucket naming rules is provided.
• Region: Specifies the data center region (e.g., sfo3 for DigitalOcean Spaces). Ensure the region matches your bucket's location.
• Endpoint: The URL endpoint for your cloud storage service. For example, https://sfo3.digitaloceanspaces.com. Leave empty if using AWS S3.

Local Copy Option

• Keep Local Copy of Uploads: Choose whether to maintain a local copy of files on the server.
  • Yes: Keeps a local version of each uploaded file.
  • No: Relies solely on cloud storage.

Action Buttons

• Sync to Cloud Synchronize files from the local server to the configured cloud storage bucket. This synchronization is also automatically run through the cron.
• Sync to Local Download files from the cloud storage bucket to the local server. This is needs to be run only manually.
• Enable/Disable Active Upload For optimal control, enable this option to allow direct uploads for file to cloud. The will reduce needs for syncing to cloud and give realtime cloud storage functionality. Note: When using this option, the module scan and patches your files (Both Perfex and modules files) where necessary to ensure direct file uploads. Thus, you need to re-enable the option when you make update to Perfex CRM or Third party modules. It also necessary after new module installation. To re-enable, simply disable and enable the feature using the same button.
• Run Tests Verifies the configuration and connectivity between the local server and cloud storage.
• Read the Documentation Opens additional resources to assist with setup or troubleshooting.

Once you've configured the necessary fields, click Save Settings to apply your changes. You will be notified of any error after saving the settings

Getting Started

1. Navigate to the Settings → Cloud Storage page.
2. Fill in the required fields:
   • Key: Obtain this from your cloud storage provider.
   • Secret: Obtain this from your cloud storage provider.
   • Bucket: Enter the name of your bucket.
   • Region: Enter the appropriate region code.
   • Endpoint: Enter the endpoint URL provided by your provider.
3. Click Run Tests to verify the setup.
4. Use the Sync buttons to upload or download files.

Bucket Naming Guidelines

Troubleshooting

Recommendations

To ensure optimal performance and security when using cloud storage services with your system, we recommend the following best practices:

• Use Versioning-Enabled Providers: We recommend selecting cloud storage providers that offer versioning, such as AWS S3, Wasabi e.t.c. Versioning ensures that you can recover previous versions of your files, offering an added layer of data protection and minimizing the risk of accidental overwrites or deletions. While DigitalOcean Spaces (like many other affordable providers) does not support versioning, consider using it in combination with other solutions to safeguard your data see this: https://docs.digitalocean.com/support/how-do-i-back-up-spaces-buckets/.
• Sync to Local or Backup Buckets: To ensure that your data is not lost in the event of an app breach (especially when using provider without versioning), you can sync your files to a local system or another cloud bucket not directly connected to your web application. This approach provides an additional backup layer, ensuring that your important files are stored securely even if your CRM app is compromised. https://docs.digitalocean.com/support/how-do-i-back-up-spaces-buckets/
• Sign Up for DigitalOcean: If you haven’t already, consider using DigitalOcean Spaces for your cloud storage. They provide reliable and cost-effective storage solutions. Use our referral link and get started with DigitalOcean today. Sign up for DigitalOcean

By following these recommendations, you can ensure that your cloud storage solution is secure, efficient, and scalable, and that your data is safely managed and easily accessible when needed.

Rescue

Active upload feature makes number of patches to both CRM files and third party modules.

Sometimes, depending on your setup, thing can possibly went wrong due to some third party modules or other environmental issues

We have prepared a secure rescue file for your use. All you need to remember is your admin login email and password:

FAQ

• Q: Can I use a custom cloud storage provider?
  A: Yes, as long as your provider supports the S3 API.
• Q: How do I ensure my files are synchronized?
  A: Synchronization to cloud is automatic via cron jobs, but you can manually trigger it using the Sync to Cloud or Sync to Local buttons. The speed of syncing is largely dependent on your storage files size and cron job interval for the automatic sync.
• Q: Can I change the bucket after initial setup?
  A: Yes, but you will need to reconfigure the module and resynchronize your files.
• Q: Are there any file size limits?
  A: File size limits depend on your cloud storage provider's policies.
• Q: How do I troubleshoot connectivity issues?
  A: Use the Run Tests feature to identify and resolve issues.

Additional Resources

• Cloud Storage Setup Guide
• Bucket Naming Rules
• DigitalOcean Spaces Documentation

Development (For Developers Only)

To avoid conflicts with older versions of dependencies used in the ‘backup’ module, it is essential to scope the dependencies under a custom namespace.

Follow the steps below to set up your development environment:

1. Ensure you have php-scoper installed globally. You can do this by running:
   composer global require humbug/php-scoper
2. Once php-scoper is installed, run the following command to install project dependencies:
   composer install

These steps will ensure the dependencies are properly scoped and prevent version conflicts.

Hooks

The module provides several hooks to customize and control folder synchronization and other related activities.

Below are some of the most important hooks:

• CLOUD_STORAGE_HOOK_SYNC_FOLDERS_FILTER - Allows you to control which folders are included or excluded from the synchronization process.
• CLOUD_STORAGE_HOOK_SYNC_FILE_FILTER - Enables you to filter individual files or groups of files during synchronization, providing flexibility to exclude unnecessary items.
• CLOUD_STORAGE_HOOK_BEFORE_SYNC - Executes custom actions or logic before the synchronization process begins.
• CLOUD_STORAGE_HOOK_AFTER_SYNC - Executes custom actions or logic after the synchronization process is completed.

For more advanced customization, explore the module's source code, particularly in the libraries folder. These files provide additional extension points and detailed insight into how each hook operates.

You can extend the library code to gain granular control over the synchronization process, ensuring the module meets your specific requirements.