Creating a cloud platform like CloudPss can be a daunting task, but it is achievable with the right knowledge and resources. Here are some general steps to help you get started:
Research CloudPss and other cloud platforms to understand their features, benefits, and drawbacks.
Define your goals and objectives for the platform. This can include features you need, target users, and any other important considerations.
Select a cloud provider and decide on the best hosting option for your platform.
Design the architecture and decide on a programming language.
Develop the platform using secure coding practices.
Test the platform for security and performance.
Deploy the platform and monitor usage.
Maintain and update the platform regularly.
Follow these steps and you will be able to create a cloud platform like CloudPss. Good luck!
what is cloudpss
CloudPss is a modeling and simulation platform for the energy internet. [1] It uses a completely self-developed electromagnetic transient simulation kernel and is designed to provide comprehensive analysis services for the energy internet. [2] It is based on cloud computing and focuses on solving the problem of modeling and simulation of power system. [3] CloudPss can be used for various purposes, including monitoring and optimization of electricity grids, power system dynamic simulation, and fault analysis.
References:
[1] Documentation - CloudPSS Documentation
[2] CloudPss-An Energy Internet Modeling and Simulation Tool in …
[3] A High-Performance Power System Simulator Based on Cloud …
how to rebuild the modeling and simulation part of cloudpss using open-source software
The modeling and simulation part of CloudPss can be rebuilt using open source software such as GridLAB-D, OpenModelica, and GridAPPS-D.
GridLAB-D is a free and open source software platform for modeling and simulation of electric power systems. It is designed to take advantage of high-performance computing resources for system analysis and optimization. [4]
OpenModelica is a free and open source modeling and simulation environment. It is designed for modeling, simulation, and optimization of complex systems. [5]
GridAPPS-D is an open-source software platform for modeling and simulation of the power system. It is designed to enable the integration of hardware and software components related to the energy internet. [6]
Using these open source software platforms is a great way to rebuild the modeling and simulation part of CloudPss.
to speed up numpy performance, either use multicore enabled BLAS library, intel MKL or use cupy (ROCm ready)
to check if numpy is not using multicore, run:
1 2 3 4 5 6
import numpy as np size = 10000 a = np.random.random_sample((size, size)) b = np.random.random_sample((size, size)) n = np.dot(a,b) # run for a long time
operations research (OR) is somehow related to machine learning and artificial intelligence. there’s a book for reference. however, a solid solver is the backbone of energy system optimization.
pyVISA is a Python package that enables you to control all kinds of measurement devices independently of the interface (e.g. GPIB, RS232, USB, Ethernet). VISA is a protocol (almost) shared by all modern hardware connection standards
in order to execute control in real time, either FPGA or RTOS like RT-Lab is used (maybe not needed?)
in simulation, unlike real-world systems, can be quite “chess-like”. you can pause the system, revert the operation, revert the time, in order to manually control all components to guide the system to optimal.
what they are talking is SCADA: supervisory control and data acquisition. SCADAS by siemens is for signal processing and recording. is it the system used in the current project? nope. is SCADAS similar to SCADA? yes.
基于改进DaNN的综合能源系统多能负荷预测, DaNN (Domain Adversarial Neural Network, Unsupervised Domain Adaptation by Backpropagation (for digital twins?)) or DDAN (Deep Domain Adaptation Networks)
Gradient Free Optimizers like: meta-heuristic, simulated-annealing, hill-climbing, particle-swarm-optimization, evolution-strategies, blackbox-optimization, gradient-free-optimization, tree-of-parzen-estimator and many more
vizier by google for blackbox and hyperparameter optimization
parmoo for parallel multiobjective simulation optimization
ilqr: Iterative Linear Quadratic Regulator with auto-differentiatiable dynamics models
RAVEN is a flexible and multi-purpose probabilistic risk analysis, validation and uncertainty quantification, parameter optimization, model reduction and data knowledge-discovering framework.
RAVEN includes the following major capabilities:
Sampling of codes for uncertainty quantification and reliability analyses
Generation and use of reduced-order models (also known as surrogate)
Data post-processing (time dependent and steady state)
Time dependent and steady state, statistical estimation and sensitivity analysis (mean, variance, sensitivity coefficients, etc.).
The RAVEN statistical analysis framework can be employed for several types of applications:
Uncertainty Quantification
Sensitivity Analysis / Regression Analysis
Probabilistic Risk and Reliability Analysis (PRA)
Data Mining Analysis
Model Optimization
VSA-FINE: generating energy system optimization models, using tsam (for Time series aggregation, Pareto-Optimal Temporal Aggregation of Energy System Models) and pyomo, compatible for both commercial solvers and open-source solvers.
python-mip (install by pip install mip) provides tools for modeling and solving Mixed-Integer Linear Programming Problems.
GEKKO is a Python package for machine learning and optimization of mixed-integer and differential algebraic equations. Modes of operation include parameter regression, data reconciliation, real-time optimization, dynamic simulation, and nonlinear predictive control. (MPC? see APM Python with Demo Applications on GitHub)
numdifftools Solves automatic numerical differentiation problems in one or more variables (Jacobian?)
NEOS is a free service for numerical optimization, which almost only accepts GAMS/AMPL/SMPS code.
COIN-OR projects (many things!) are usually solver and optimization related. Pyomo (a collection of Python software packages for formulating optimization models, similar to linopy (only for labeled data, linear optimization)) is part of the project.
OpenSolvers is for microsoft excel and uses linear solvers like COIN-OR CBC (on which mentioned a lot of modeling languages/tools it supports) and non-linear solvers like Bonmin (Basic Open-source Nonlinear Mixed INteger programming) and Couenne (Convex Over and Under ENvelopes for Nonlinear Estimation) powered by IPOPT. details on solvers
flux.jl is machine/deep learning in julia, with GPU support. model zoo (examples) and doc. DiffEqFlux.jl can add differential equation layers to neural networks, details here
microsoft machine teaching is like automl for reinforcement learning (autorl), based on bonsai (on [github[(https://github.com/BonsaiAI)) (doc) is a low-code platform for autonomous AI systems (bonsai BRAIN is close-sourced), doc before acquiantaince. mentioned “optimize HVAC (heating, ventilation, and air conditioning) with EnergyPlus and BCVTB (Building Controls Virtual Test Bed)”. specify enviorments, rewards and targets with Inkling (declarative)
NTM (neural turing machine) (best?) for translation, question answering, planning and decision-making. tensorflow 1.14.0 includes it into tf.contrib which also has timeseries module.
model predictive control
MPC is mentioned along with PID (Proportional Integral Derivative) found in libraries like simple-pid, pypid and control.
gradcem: Model-Predictive Control via Cross-Entropy and Gradient-Based Optimization
do-mpc (in python) is a comprehensive open-source toolbox for robust model predictive control (MPC) and moving horizon estimation (MHE).
mpc-pytorch is a fast PyTorch library for solving the non-convex control problem
modeling & simulation
for interfaces i’d like jupyter. does julia/modelica/octave support jupyter?
thermalcorr: (c++) A collection of thermohydraulic correlations for thermal systems and heat exchangers
neural operator for PDE/ODE neural network approximation (speedup solving) and doc
hydraulic: A python package to compute pressure, flows and temperatures in hydraulic circuits
fluids: is open-source software for engineers and technicians working in the fields of chemical, mechanical, or civil engineering. It includes modules for piping, fittings, pumps, tanks, compressible flow, open-channel flow, atmospheric properties, solar properties, particle size distributions, two phase flow, friction factors, control valves, orifice plates and other flow meters, ejectors, relief valves, and more.
CityLearn: reinforcement learning environment for controlling the storage of domestic hot water (DHW), chilled water (for sensible cooling and dehumidification) hot water (for sensible heating) and electricity, also including models of air-to-water heat pumps, electric heaters, solar photovoltaic arrays, and the pre-computed energy loads of the buildings, which include space cooling, dehumidification, appliances, DHW, and solar generation.
OMF: The Open Modeling Framework for smart grid cost-benefit analysis.
HELICS is a co-simulation framework, designed to bring together domain-specific modeling tools so they interact during run time. usage examples and doc
PLEXOS by Energy Exemplar is the only solution available today that unifies market simulations across the electric, water, and gas industries.
ModelicaGym: Modelica models integration with Open AI Gym
andes: Python toolbox / library for power system transient dynamics simulation with symbolic modeling and numerical analysis
DPSim: Real-time power system simulator including powerflow, (dynamic) phasors and EMT. doc
calliope: A multi-scale energy systems modelling framework
Thermal Engineering Systems in Python (TESPy). This package provides a powerful simulation toolkit for thermal engineering plants such as power plants, district heating systems or heat pumps.
Energy-Hub-Market-Operation: Participation of an Energy Hub in Electricity and Heat Distribution Markets using MPEC (mathematic program with equilibrium constraints)
fmi (functional mock-up interface) is an open model exchange format that able to run on multiple languages, proposed by modelica. tools support this standard are usually for modeling and simulation, and are able to communicate with each other using this protocol.
TRNSYS(Transient System Simulation Program, paid, not open source)最早是由美国威斯康星大学和Solar Energy 实验室开发的。容易与MATLAB、python、C++进行调用和编程。主要进行建筑负荷模拟相对于DEST来说较为复杂。可进行太阳能（光热、光电）、地源热泵、风电、光电、氢能、三联供、综合能源、全生命周期经济性系统模拟分析
multisim includes models like pipes, heat exchanger, thermal storage and controllers like PID, Bang–bang controller,
Two sensor controller and Model predictive controller.
Tools for Energy Model Optimization and Analysis (TEMOA) is an open source modeling framework for conducting energy system analysis. The energy system is described algebraically as a network of linked processes that convert a raw energy commodity (e.g., coal, oil, biomass, uranium) into an end-use demand (e.g., lighting, transport, water heating, conditioned air), doc is here
Distributed Energy Generation Model: Code for “Natural Gas Combined Cycle Power Plants Have Lower Environmental Impact than Conventional Combined Heat and Power for Commercial Buildings, Environmental Science & Technology”
flixOpt: vector based energy and material flow optimization framework in python
Wastewater_Energy_Optimization: Supplementary files for “Integrated Design and Optimization of Water-Energy Nexus: Combining Wastewater Treatment and Energy System”
GAN_GA_ICML containing files to replicate experiments and results from “Using C-GANs to Boost High-dimensional Nonlinear Optimization for Sustainability Targets”
IDEAS (Modelica library allowing simultaneous transient simulation of thermal and electrical systems at both building and feeder level.
IESLAB for burst and leakage simulation/detection with SCADA data
Grid2Op: a testbed platform to model sequential decision making in power systems (for Reinforcement Learning?), with lightsim2grid which implements a c++ backend targeting the Grid2Op (speedup simulation). doc
The Framework for Optimization of ResourCes and Economics (FORCE) is a collection of software tools developed under the Integrated Energy Systems (IES) program to enable analysis of technical and economic viability of myriad IES configurations. Each of these tools is openly available and free to use.
HERON: Holistic Energy Resource Optimization Network (HERON, for Technoeconomic Assessments and Synthetic History Generation) is a modeling toolset and plugin for RAVEN to accelerate stochastic technoeconomic assessment of the economic viability of various grid-energy system configurations, especially with application to electrical grids and integrated energy systems (IES).
TRANSFORM-Library: A Modelica based library for modeling thermal hydraulic energy systems and other multi-physics systems
smart-rl-mg: Reinforcement Learning + Microgrids for OpenDSS
HYBRID (requires DYMOLA? just a Modelica runtime?) is a modeling toolset to assess the integration and economic viability of Integrated Energy Systems (IES) , including Modeling and Experimental Validation of Latent Heat Thermal Energy Storage System. Related to CENTAUR which has a GUI interface (very abstract), can be used for the design and simulation of hybrid solar PV-battery-diesel microgrids , similar to (only simulation, not optimization) HYBRID2 and HOMER (Hybrid Optimization Model for Multiple Energy Resources, with a data processing script found somewhere)
EMSO is the acronym for Environment for Modeling, Simulation, and Optimization. can customize model using EMSO modeling language. it is about chemistry rather than CCHP according to its modeling library EML
OpenModelica is an open-source Modelica-based1 modeling and simulation environment intended for industrial and academic usage.
Xcos by SciLab is a graphical editor to design hybrid dynamical systems models. Models (in modelica) can be designed, loaded, saved, compiled and simulated. All Xcos standard blocks grouped by categories (signal processing, electrical, hydraulics, derivative, integral, etc.).
JModelica is an extensible Modelica-based open-source platform for optimization, simulation, and analysis of complex dynamic systems. The main objective of the project is to create an industrially viable open-source platform for simulation and optimization of Modelica models.
Scilab (by Dassault Systems) is a scientific software package for numerical computations providing a powerful open computing environment for engineering and scientific applications. Scilab is an open source software and includes hundreds of mathematical functions with the possibility to add interactively programs from various languages (C, C++, Fortran…).
Scicos is a graphical dynamical system modeler and simulator developed in the Metalau project at INRIA, Paris-Rocquencourt center. With Scicos, user can create block diagrams to model and simulate the dynamics of hybrid dynamical systems and compile models into executable code.
Spyder is a free open-source Python development environment providing MATLAB-like features in a simple and light-weighted software.
StaticDESim: Simulation and optimisation of integrated energy system in steady state written in MATLAB, with models like Micro Gas Turbine (MGT), Aqueous Lithium-Bromine Single-Effect Absorption Chiller (AC_ALB), R123 Organic Recycle Cycle (ORC_R123), Heat Pump (HP) and R134a Vapour Compression Chiller (VCC_R134a)
OpenHydraulics: A free Modelica package providing components describing 1-dimensional fluid flow in hydraulic circuits.
pyDMPC: A Python tool for distributed model-based predictive control of energy suppy chains
Ai4EMetaPSE by Ai4Energy: 采用基于方程的面向对象建模，基于微分代数方程（DAE），对能源系统组件进行建模，用以对能源系统进行稳态、动态仿真。能够处理连续时间及离散事件问题。
simupy: a ramework for modeling and simulating dynamical systems
Modelica Standard Library includes models for mechanical (1D/3D), electrical (analog, digital, machines), magnetic, thermal, fluid, control systems and hierarchical state machines, similar to Modeling Toolkit Standard Library from SciML
cchp-minlp models including heat pump, gas turbine, auxiliary boilers, written in GAMS and AMPL
pymgrid only provides models related to electricity.
oemof: Open Energy Modelling Framework - Python toolbox for energy system modelling and optimization, with thermal simulation support. doc
renpass: Renewable Energy Pathways Simulation System (written in R)
MESSAGEix is a framework written in GAMS that can be used to develop and run many different models, each describing a different energy system.
Minpower is an open source toolkit for students and researchers in power systems.
PandaThermal: District heating and cooling network design and simulation tool, largely inspired by PandaPower
SciML is Scientific Computing + Machine Learning Toolbox. It has the best performant DifferentialEquations.jl (can run on CUDA with ease, connect to neural networks) and NeuralPDE.jl for Physics-Informed Neural Networks (PINN) and Deep BSDE Solvers of Differential Equations for Scientific Machine Learning (SciML) accelerated simulation, easily bridging to Python. In its tutorial we can learn differential equation modeling/simulation.
PowerSimulations.jl is a Julia package for power system modeling and simulation of Power Systems operations.
TIMES model: The Integrated MARKAL-EFOM System, written in GAMS (General Algebraic Modelling System, a modeling language, paid, similar to Dyna (a declarative modeling language like prolog, in dyna2gams it can refine parameters on observation) with lots of solvers), able to model CHS systems and predict financial costs said in its documendation, is a technology rich, bottom-up model generator, which uses linear-programming to produce a least-cost energy system, optimized according to a number of user constraints, over medium to long-term time horizons.
julia relates quite a bit to power and energy simulations. is it performance related?
type ] in julia’s REPL to get Pkg interface.
Modelica is a language for modeling of cyber-physical systems, supporting acausal connection of components governed by mathematical equations to facilitate modeling from first principles, used in CATIA Systems, Dymola, JModelica.org, LMS AMESim, MapleSim, Modelon Impact, MWorks, OpenModelica (for python you need this: PySimulator), SimulationX, and Wolfram SystemModeler (and more tools.
OpenIPSL is a library of power system component models written in the Modelica language that can be used for power system dynamic analysis, such as phasor time-domain simulations.
STEPS & stepspy: Simulation Toolkit for Electrical Power Systems. doc
pandapower: (lacks storage unit models) Convenient Power System Modelling and Analysis based on PYPOWER and pandas. official site
in pandapower’s paper there are lots of open source tools for electric power systems optimization mentioned:
MATPOWER (with optional solvers, must be obtained separately) is a widely used power system analysis tool for matlab and octave that solves power flow and optimal power flow problems and also includes an optimal scheduling tool for market simulations.
There are ports of the original MATLAB based code to other languages, most notably Pythons PYPOWERpypower-dynamics extends PYPOWER for dynamic analysis of electric power systems.
GridCal (abbriviation for grid calculator) includes power flow, time-series and short circuit calculation methods and comes with a comprehensive graphical user interface.
The simulation and optimization library PyPSA (Python for power system analysis) is aimed at time-series simulation of security-constrained linear optimal power flow and investment optimization.
PowerGAMA ( Python-based lightweight simulation tool for high level analyses of renewable energy integration in large power systems) and psst (power system simulation toolbox) are also aimed at market optimization in electric networks.
GridLAB-D is an advanced tool for static simulations using agent based simulation models, based on C/C++ with integration of SCADA controls, metering and market models.
OpenDSS is a electrical power system simulation tool primarily for utility power distribution systems, written in a Delphi.
MATPower: free, open-source tools for electric power system simulation and optimization
energyplus (open source, based on DOE2, a building energy analysis program that can predict the energy use and cost) is a whole building energy simulation program that engineers, architects, and researchers use to model both energy consumption—for heating, cooling, ventilation, lighting and plug and process loads—and water use in buildings.
bcvtb is a software environment that allows users to couple different simulation programs for co-simulation, and to couple simulation programs with actual hardware. For example, the BCVTB allows to simulate a building in EnergyPlus and the HVAC and control system in Modelica, while exchanging data between the software as they simulate.
in SimStudio it has “octave” keyword. does that mean it is using GNU octave? yes! but octave does not have Simulink-like functionalities. see octave packages for more. use Xcos instead? in order to read code in modules like https://cloudpss.net/model/CloudPSS/HeatPump(右键点击元件，选择打开模块) you have to obtain sufficient permission.
will raise error if we wire electricity to water pipelines. it is not so easy to wire these things. better do it with port routing.
DC Current Source, DC Voltage Source, Single-phase AC Voltage Source, Controlled Current Source, Controlled Voltage Source, Controlled AC Voltage Source (VF), Controlled AC Voltage Source (VP)
Electrical-Power Electronic Switches
Diode, Thyristor, IGBT
Electrical-Three-phase Components
Shunt Capacitor/Reactor, Fixed Load, Line Cluster, Three-phase Transmission Line, Three-phase AC Bus, Three-phase AC Voltage Source, Three-phase Two-winding Transformer, Three-phase Three-winding Transformer, Synchronous Generator
Electrical-Fast Power Electronic Modules
Half Bridge Submodule, Six-pulse Thyristor Bridge
Electrical-Renewable Energy Components
Photovoltaic Source, Lead-acid Battery
Measure Components
Branch Voltage Meter, Voltage Meter, Current Meter, RMS Meter, Three-phase Power Meter, Phase Locked Loop, FFT
Control-Basic Components
M script, Constant, Time, Simulation Time Step, Non Connection (NC), Loop Break Node, Channel Merge, Channel DeMerge
Control-Basic Math Functions
Adder/Subtractor, Multiplier, Divider, Absolute Value, Sign Function, Round Function, Trigonometric Function, Power Function, Exponential Function, Logarithm Function, Maximum/Minimum Function, Maximum/Minimum in One Cycle
Control-Linear Transfer Functions
Gain, Integrator, Derivative, PI Controller, Zero-point, Real Pole, Differential Pole, Lead Lag Pole, Second Order Complex Pole, Nth Transfer Function
Control-Nonlinear Functions
Hard Limiter, Delay, Angle Resolver, Piecewise Linear Function, Nonlinear Function
Control-Analog Signal
Comparator, Hysteresis Comparator, Zero Detector, Sampler, Sample and Hold
Park Transformation, Clark Transformation, dq-αβ Coordinates Transformation, Polar/Rectangular Coordinate Converter
Control-Singal Generator
Triangular Generator, Squar Generator, Sine Generator, Adjustable FPM Sine Generator, Single-impulse Generator, Impulse Generator, Step Generator, Ramp Generator, Surge Gennerator, Drop Generator, Random Number Generator
Pipeline, Heat Source, Bulding (Load), Connection Point, Relay Pump
IESLab (Integrated Energy System): 综合能源系统规划设计云平台
a place where you can import weather, energy prices, device parameters and loads data, design the IES system by hand (how components connects), determine the best products to purchase based on different criterions and assess it by net profit, environmental effects and energy efficiency.
FuncStudio: 助力能源电力系统数字孪生云边融合业务定制
install locally (windows only?) or download binary file of “header only” functions?
AppStudio: 快捷构建能源电力系统数字孪生应用
low-code frontend design interface.
the platform uses code generator to make the simulation fast.
something like siemens simcenter is also cloud based. collimator is self-labeled as better simulink alternative with ai and python code blocks inside. it also suggests model based development (and a hell lots of blogs) for a general guideline on simulation and optimized control systems.
cloudpss is utilizing alibaba’s cloud computing. the simulation uses computing graph and cuda acceleration (diffcult to deploy?), suggested simulink and PSCAD (PyPSCAD does similar things, but far from complete) (with IncrediBuild-XGE for multi-core acceleration) for simulation. in the paper it describes itself as “power system simulator” and “electromagnetic transient simulator” (HVDC: high voltage direct current, highly efficient for transmitting large amounts of electricity over long distances)
set up python with appropriate version on client’s computer by script, not by “PyInstaller” which takes huge amount of time to compile and huge disk space (visually).
film.ai now can query screenshot and movie name by description and download thumbnails of movies (not latest, not mainland), but without subscription you cannot get accurate seek time (though it will never be accurate)
in imdb can pass film/anime name in multiple languages and get the english name (and trailer video), then query for it in 1337x (results sorted by seeder counts)
文本分类式的番剧剪辑 需要分割时间段 即每隔一分钟分割对应的弹幕并摘要或者字幕 合并并进行打标签 训练 注意不要包括片头和片尾 (maybe audio only model like whisper will classify this successifully, remember to split (or not?) vocals from BGM? (to detect singing voice which is unique in OP/ED))
when transcoding (with seeking?) using ffmpeg tweak parameters. set low profile with high threads count (higher crf will result in poor quality but faster speed), although all these flags may result into unplayable video for some players.
aria2c can be controlled via python (to make sure it will exit immediately after finishing download instead of seeding and blocking, though can be achieved with some tweaks on commandline arguments to execute command after download finished signal emitted): aria2p (can be used both as a library or cli program), pyaria2 (old) searching aria2 in github, i found some repos relating to baidunetdisk.
如果要实时看云电脑的进度可以自己搭建一个netprogressbar server 根据约定好的url和密码 (read-only and write-only password, or both, by setting different privilege) 来上报和接收进度 server要及时回收资源