D3T: Double Deep Q-Network Decision Transformer for Service Function Chain Placement

Network Function Virtualization (NFV) has become a promising technology, which is used to replace the complex hardware implementation of various Network Functions. Virtual Network Functions (VNFs) are placed on the servers to realize these functions of a network, for example 5G network. Such an implementation reduces the expenditure and latency significantly. However, it also comes with its own set of challenges. One of the prominent challenges of NFV is the placement problem. The requests are in the form of a Service Function Chain (SFC), which is a pre-defined sequence of VNFs. When an SFC request comes in, resources must be allocated to the VNF instances to satisfy the requirements. In this paper, we propose an effective algorithm D3T (Double Deep Q-Network Decision Transformer) to optimize the SFC placement. The algorithm is designed using a Decision Transformer (DT) that is assisted by a Double Deep Q-Network (DDQN). We employ a DDQN model as the baseline algorithm to generate offline training data. The trajectory data in the Experience Reply Memory of DDQN will be processed into sequences and modeled by a transformer. The algorithm’s objective function considers end-to-end delay and rejection ratio as the objectives. Specifically, D3T combines the transformer with DRL to give an optimal solution. The results presented demonstrate the effectiveness of the proposed solution.