Research on Distributed Storage Energy-saving Technology Based on Ceph
Research on Advanced Computing and Data Processing Based on Ceph's Distributed Storage Energy-Saving Technology SHEN Fan, WU Qingbo, YANG Shazhou (School of Computer Science, National University of Defense Technology, Changsha 410073). For this reason, based on Ceph distributed storage, analysis of its data layout in the energy-saving deficiencies, the proposed energy-saving optimization algorithm to divide the power group, in order to improve the system energy-saving ratio. Establish a multi-level power consumption model of Ceph and give a management strategy, design and implement a multi-level power management framework of Ceph system for dynamic management of Ceph system power consumption. 3.2 Insufficiency of storage system In many application scenarios, there is a low load At this time, in order to save the energy consumption of the system, some data nodes can be shut down. For the following considerations, close the node should ensure that the data set is available (that is, at least more than one copy of any data object is not closed): On the other hand, if access to unavailable data occurs, very large accesses will be generated. Delay; On the other hand, frequent opening of the corresponding node will generate extra energy that cannot be ignored. In a Ceph storage system, the number of data nodes is N is divided into n fault domains fd, and the replica distribution policy is select(r,/d). Thus, the number of fault domains that can be closed at most during the system low load system is n. ' Optimized data layout At this point, there are 2 nodes that can be closed and all data is still available. In the optimized data layout, the copies of the data are located in r different power consumption groups. When the data set is available, the nodes of the r-1 power consumption group can be turned off at most. The energy saving ratio that the system can achieve can be achieved. For r-1/r, when the cluster size is large, the energy consumption saved is very considerable. At the same time, due to the good distribution of replicas, the number of power groups that can be turned off can be any one, thus providing a basis for multi-level power management of the system. 4 Multi-level power management Based on the optimization of data copy distribution, the number of active power groups in the system can be adjusted as needed according to the system I/O load conditions, so that the system is at different power consumption levels, thus achieving the system's Multi-level power management reduces power consumption. 4.1 Multi-level power model In Ceph, the main energy consumption of the system comes from the OSD nodes. In a Ceph cluster with n OSD nodes, the number of replicas is set to r. After the data layout is optimized, the nodes are divided into r power groups. If the power consumption of a single OSD node is p, then the power consumption of a single power group The power consumption is: If the number of active (non-closed or dormant) power groups in the system is equal to the system power consumption, the system power consumption is: where rac can be 1 ~ r, that is, the system can be in P2, Pr different power consumption. level. During a period of time T, the energy consumed by the system is: where is the time that the system is at the corresponding power level; it is the sum of the energy consumed by the system for level switching. 4.2 Power Level Management The main task of managing power level management is to dynamically adjust the power level according to the I/O load status, while minimizing power consumption while ensuring service quality. I/O load status can be determined through statistical analysis or prediction. The former is used to collect and count I/O data such as I/O times, I/O data volumes, etc., within a certain period of time. This determines the system's I/O load status. To describe the I/O load status in different scenarios, both random I/O and sequential I/O need to be collected and counted. Therefore, in the configurable time window W, the system's I/O status data can be counted. It is: Among them, Ortotal is the number of random I/Os that occur in W time; lOstotal is the amount of data that the sequential I/O requests in W time. The two are respectively compared with the pre-measured peak I/O capability of the system to obtain a quantitative description of the I/O status of the system, that is, the I/O load ratio L: where, it is used to compare with the current power consumption rate Pi=ractr. Determine the power level in the next stage of the system: When the system's I/O load rate is higher than the energy consumption rate, the active power consumption group can no longer meet the requirements of the I/O load, and more power consumption groups are required to provide services. When the system's I/O load rate is lower than the energy consumption rate, some power consumption groups in the system can be turned off to achieve energy saving. The pseudo-code determined by the power level is: the number of power levels in the system. Computer Engineering The racte output on August 15, 2015 will be used as the power level of the system in the next W period, and needs to be turned off/off. The number of power groups that are turned on is I-I. To ensure high availability of the system, you can set the minimum number of active power groups that are allowed to 2, that is, allow at least two copies of system data to be available. 5 system implementation and, the power management framework consists of 4 modules, including LayoutOptimizer module to achieve the optimization algorithm for CRUSH data layout, and generate new Crushmap and placement rules; I/OTracer module for tracking and statistical system I /O data, that is, collect and analyze the I/O related information records in CephLog at a certain frequency; LeverShifter is a module that manages the power level of the system, and analyzes the I/O load of the current system according to the I/O data that is tracked. State, based on the selected strategy to determine whether the need to switch power levels; StatusManager module is responsible for the implementation of power level switching, first use Ceph OSD state management tool to set the state of the OSD Ceph cluster to noout, guaranteed not to be active Stop the OSD and data migration occurs. Then use technologies such as Remote Sleep/Wake-on-LAN (WOL11) to control the server power management status of the OSD. The sleep/wake-up method is more energy-efficient than the traditional method of shutting down/opening the server. The number of times each energy level was used. When the energy-saving framework is not used, the energy consumption of the system running for 8 hours is about 48x6x10minx400W. After the energy-saving framework (Ceph-PM) is turned on, the energy consumption of the system running for 8h is about (11x2+12x4+25x6)x10minx400W, and the system achieves energy saving. The proportion is about 25%. If the scale of the system is increased, the saved energy expenditure will be very considerable. However, it should be noted that in the real environment, the system load may change more frequently and violently. Therefore, a more accurate and complicated load level switching strategy is required. This is one of the contents of this article in the future. With the average response time of sequential I/Os and write operations, the number of times to write copies is low for low power consumption. However, for the client, the response time is smaller, and the effect on sequential writes is more pronounced. Shen Haoji, Wu Qingbo, Yang Shazhou: Research on Ceph-based distributed storage energy-saving technology 6 Conclusion This article is based on the Ceph system, studies distributed storage technologies, analyzes the insufficiency of data layout based on CRUSH algorithm, and proposes an optimization algorithm for energy saving purposes. System multi-level power management strategy, and achieved Ceph's multi-level power management framework. The experimental results show that the energy management framework can dynamically adjust the system power consumption level according to the system load changes and effectively reduce the system energy consumption.
High efficiency surface grinding or fine grinding is a super finishing process.
Applicated in the fields:
Blade pump, steering valve, plunger pump, gasoline pump,hydraulic motor, etc.
Air condition compressor blade, piston, cylinder, refrigerator compressor valve plate, etc.
Internal combustor part, booster pump part, oil nozzle/ oil pump part, etc Various hard alloy blades, high precision bearing, sealing piece, watch part, ceramic valve core, magnetic material, etc.
GMM700 Double Side Surface Grinding Machine Precision Surface Grinding Machine,Double Side Surface Grinder,Stainless Steel Surface Grinding Machine,Batch Surface Grinding Machine Xinxiang SKF machinery Co., Ltd. , https://www.grindingmachine.nl
