API tools faq
paste
Advertisement
NMamaev

Untitled

NMamaev
Apr 10th, 2023
55
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Bash 66.49 KB | None | 0 0
raw download clone embed print report
  1. [root@1410738da7ed log]# grep a78798d39ec883f8ce3cccd39a58b983 *
  2. connect_storages.log:2023-04-10 06:28:56.460 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] INFO Try connect to http://input:1500/vm/v3/node/632 with headers {'Host': 'instance-1', 'internal-auth': 'on'}
  3. connect_storages.log:2023-04-10 06:28:56.461 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG Starting new HTTP connection (1): input:1500
  4. connect_storages.log:2023-04-10 06:28:56.465 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG http://input:1500 "GET /vm/v3/node/632 HTTP/1.1" 200 1533
  5. connect_storages.log:2023-04-10 06:28:56.522 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] INFO Try connect to http://input:1500/vm/v3/node/632/storage/509/connect_params with headers {'Host': 'instance-1', 'internal-auth': 'on'}
  6. connect_storages.log:2023-04-10 06:28:56.523 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG Starting new HTTP connection (1): input:1500
  7. connect_storages.log:2023-04-10 06:28:56.528 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG http://input:1500 "GET /vm/v3/node/632/storage/509/connect_params HTTP/1.1" 200 202
  8. connect_storages.log:2023-04-10 06:28:56.528 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run pvscan --cache && pvscan && vgscan && lvscan
  9. connect_storages.log:2023-04-10 06:28:56.769 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run pvs --reportformat json -o vg_name /dev/sda
  10. connect_storages.log:2023-04-10 06:28:56.868 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] ERROR pvs --reportformat json -o vg_name /dev/sda failed with code 5 stderr "  Failed to find physical volume "/dev/sda".
  11. connect_storages.log:2023-04-10 06:28:56.869 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run pvcreate /dev/sda -qq
  12. connect_storages.log:2023-04-10 06:28:56.969 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run vgcreate Network_lvm_dev_sda /dev/sda
  13. connect_storages.log:2023-04-10 06:28:57.024 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] ERROR vgcreate Network_lvm_dev_sda /dev/sda failed with code 5 stderr "  A volume group called Network_lvm_dev_sda already exists.
  14. connect_storages.log:2023-04-10 06:28:57.024 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] ERROR failed to connect storage with id "509" error "Command exited with message   A volume group called Network_lvm_dev_sda already exists.
  15. connect_storages.log:2023-04-10 06:28:57.025 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] INFO Try connect to http://input:1500/vm/v3/node/632/storage/510/connect_params with headers {'Host': 'instance-1', 'internal-auth': 'on'}
  16. connect_storages.log:2023-04-10 06:28:57.026 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG Starting new HTTP connection (1): input:1500
  17. connect_storages.log:2023-04-10 06:28:57.030 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG http://input:1500 "GET /vm/v3/node/632/storage/510/connect_params HTTP/1.1" 200 202
  18. connect_storages.log:2023-04-10 06:28:57.030 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run pvscan --cache && pvscan && vgscan && lvscan
  19. connect_storages.log:2023-04-10 06:28:57.244 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run pvs --reportformat json -o vg_name /dev/sdb
  20. connect_storages.log:2023-04-10 06:28:57.300 [pid=3313] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] ERROR failed to connect storage with id "510" error "physical vollume "/dev/sdb" is busy, unknown volume group "Network_lvm_dev_sda" was foun"
  21. ifacewatch.log:2023/04/10 06:28:56 [pid=108] Web New request: "instance-1" - "POST" - "/watcher" - http.Header{"Connection":[]string{"close"}, "Content-Length":[]string{"53"}, "Global-Attributes":[]string{"{\"callback_counter\":36,\"first_start\":1817893060963609,\"request\":{\"id\":\"a78798d39ec883f8ce3cccd39a58b983\",\"ip\":\"172.19.0.12\",\"user\":{\"name\":\"n.mamaev@ispsystem.com\"}},\"runtime\":162981907618}"}, "Instance-Id":[]string{"1"}, "Internal-Auth":[]string{"on"}, "X-Forwarded-For":[]string{"172.19.0.12"}, "X-Remote-Ip":[]string{"172.19.0.12"}, "X-Request-Id":[]string{"a56c4eb3cc03518e4912f33de41bb9a6"}}
  22. grep: nginx: Is a directory
  23. node_configure_spice.log:2023-04-10 06:28:54.326 [pid=3286] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] INFO Enable node spice
  24. node_configure_spice.log:2023-04-10 06:28:54.327 [pid=3286] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run sed -i 's/^[# ]*spice_listen.*/spice_listen = "172.31.36.34"/g' /etc/libvirt/qemu.conf
  25. node_configure_spice.log:2023-04-10 06:28:54.383 [pid=3286] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run systemctl restart libvirtd
  26. node_network.log:2023-04-10 06:28:53.001 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] INFO node network configure
  27. node_network.log:2023-04-10 06:28:53.053 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] INFO Change default network forward mode
  28. node_network.log:2023-04-10 06:28:53.053 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run EDITOR='sed -i "s/nat/open/g"' virsh net-edit default
  29. node_network.log:2023-04-10 06:28:53.147 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run virsh net-start default
  30. node_network.log:2023-04-10 06:28:53.191 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG checking if vmbr0 is on None
  31. node_network.log:2023-04-10 06:28:53.191 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run ip link show vmbr0
  32. node_network.log:2023-04-10 06:28:53.197 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] ERROR ip link show vmbr0 failed with code 1 stderr "Device "vmbr0" does not exist.
  33. node_network.log:2023-04-10 06:28:53.198 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run ip route get 1 | awk '{print $5;exit}'
  34. node_network.log:2023-04-10 06:28:53.243 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run cat /etc/network/interfaces
  35. node_network.log:2023-04-10 06:28:53.247 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG found iface eth0
  36. node_network.log:2023-04-10 06:28:53.247 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG starting to get info about iface eth0
  37. node_network.log:2023-04-10 06:28:53.247 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run cat /proc/net/route | grep eth0
  38. node_network.log:2023-04-10 06:28:53.252 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run cat /proc/net/route | grep eth0
  39. node_network.log:2023-04-10 06:28:53.299 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG Checks line: 00000000
  40. node_network.log:2023-04-10 06:28:53.299 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG it is default
  41. node_network.log:2023-04-10 06:28:53.299 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG and have gateway
  42. node_network.log:2023-04-10 06:28:53.299 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG gateway is 172.31.36.1
  43. node_network.log:2023-04-10 06:28:53.299 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] INFO basic interface check
  44. node_network.log:2023-04-10 06:28:53.300 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run test -d /sys/class/net/eth0
  45. node_network.log:2023-04-10 06:28:53.302 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run test -d /sys/class/net/eth0/bonding_slave
  46. node_network.log:2023-04-10 06:28:53.305 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] ERROR test -d /sys/class/net/eth0/bonding_slave failed with code 1 stderr ""
  47. node_network.log:2023-04-10 06:28:53.305 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run test -d /sys/class/net/eth0/bridge
  48. node_network.log:2023-04-10 06:28:53.350 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] ERROR test -d /sys/class/net/eth0/bridge failed with code 1 stderr ""
  49. node_network.log:2023-04-10 06:28:53.350 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run test -d /sys/class/net/eth0/brport
  50. node_network.log:2023-04-10 06:28:53.353 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] ERROR test -d /sys/class/net/eth0/brport failed with code 1 stderr ""
  51. node_network.log:2023-04-10 06:28:53.353 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] INFO common DC bridge live configuring
  52. node_network.log:2023-04-10 06:28:53.359 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG getting netmask on interface eth0
  53. node_network.log:2023-04-10 06:28:53.360 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run ip a show dev eth0 | grep "inet " | awk '{print $2;exit}'
  54. node_network.log:2023-04-10 06:28:53.365 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG mask is 23
  55. node_network.log:2023-04-10 06:28:53.365 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run test -d /tmp
  56. node_network.log:2023-04-10 06:28:53.414 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run touch /tmp/vm.3ci4Svhwtz
  57. node_network.log:2023-04-10 06:28:53.417 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG make_temp tmp: /tmp/vm.3ci4Svhwtz make_temp data: #!/bin/sh
  58. node_network.log:2023-04-10 06:28:53.421 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run /bin/bash /tmp/vm.3ci4Svhwtz
  59. node_network.log:2023-04-10 06:28:53.432 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run cat /etc/network/interfaces
  60. node_network.log:2023-04-10 06:28:53.444 [pid=3274] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG configure network with params:
  61. node_network.log:2023-04-10 06:28:53.851 [pid=3283] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] INFO get node vnc address
  62. node_network.log:2023-04-10 06:28:53.905 [pid=3283] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG run ip route get 1 | awk '{print $7;exit}'
  63. node_network.log:2023-04-10 06:28:53.960 [pid=3283] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] DEBUG vnc address 172.31.36.34
  64. vm_1_reader.log:2023-04-10 06:31:48.304 [pid=217] [#7667f8fae8c1b7e451539973076aa590/10.60.4.173/@n.mamaev@ispsystem.com] Response body '{"last_notify":5432922,"task":{"account":{"email":"n.mamaev@ispsystem.com","id":4,"ip":"172.19.0.12"},"date_create":"2023-04-10 06:26:13.000000","date_end":"2023-04-10 06:28:57.000000","date_start":"2023-04-10 06:28:56.000000","message_params":{"name":"HA_Node_1"},"name":"connect_storages","output":"{\"before_execute\":{\"last_notify\":5432041},\"storages\":[{\"error\":\"Command exited with message   A volume group called Network_lvm_dev_sda already exists.\\n.\",\"id\":509},{\"error\":\"physical vollume \\\"/dev/sdb\\\" is busy, unknown volume group \\\"Network_lvm_dev_sda\\\" was foun\",\"id\":510}]}","param":["--action","connect_storages","--virt-type","kvm","--params","{\"cluster_id\":22,\"node_id\":632,\"node_ip\":\"172.31.36.34\",\"node_ssh_port\":22,\"storages\":[{\"id\":509,\"type\":\"network_lvm\"},{\"id\":510,\"type\":\"network_lvm\"}]}"],"request_id":"a78798d39ec883f8ce3cccd39a58b983","status":"failed"}}'
  65. vm_1_reader.log:2023-04-10 06:33:23.566 [pid=217] [#8045ad5d395b017d73fe1b9dacdbc49e/10.60.4.173/@n.mamaev@ispsystem.com] Response body '{"last_notify":5433333,"task":{"account":{"email":"n.mamaev@ispsystem.com","id":4,"ip":"172.19.0.12"},"date_create":"2023-04-10 06:26:13.000000","date_end":"2023-04-10 06:28:57.000000","date_start":"2023-04-10 06:28:56.000000","message_params":{"name":"HA_Node_1"},"name":"connect_storages","output":"{\"before_execute\":{\"last_notify\":5432041},\"storages\":[{\"error\":\"Command exited with message   A volume group called Network_lvm_dev_sda already exists.\\n.\",\"id\":509},{\"error\":\"physical vollume \\\"/dev/sdb\\\" is busy, unknown volume group \\\"Network_lvm_dev_sda\\\" was foun\",\"id\":510}]}","param":["--action","connect_storages","--virt-type","kvm","--params","{\"cluster_id\":22,\"node_id\":632,\"node_ip\":\"172.31.36.34\",\"node_ssh_port\":22,\"storages\":[{\"id\":509,\"type\":\"network_lvm\"},{\"id\":510,\"type\":\"network_lvm\"}]}"],"request_id":"a78798d39ec883f8ce3cccd39a58b983","status":"failed"}}'
  66. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] request method: POST url: /node version: HTTP/1.0
  67. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Handler 'node' choosen
  68. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Got header: [connection]close
  69. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Got header: [content-length]244
  70. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Got header: [es-extra-tags]batch.0c1c5516-8883-f662-489c-821bd04a167a
  71. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Got header: [host]instance-1
  72. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Got header: [instance-id]1
  73. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Got header: [internal-auth]eyJlbWFpbF9jb25maXJtZWQiOnRydWUsImV4cGlyZXNfYXQiOiIiLCJpbnN0YW5jZV9vd25lciI6ZmFsc2UsImxhbmciOiJydSIsImxhc3Rfbm90aWZ5Ijo1NDMxMjY4LCJvd25lcl9lbWFpbCI6Im4ubWFtYWV2QGlzcHN5c3RlbS5jb20iLCJvd25lcl9pZCI6NCwicm9sZXMiOlsiQGFkbWluIl0sInNlc19pZCI6MTI5Miwic2Vzc2lvbiI6IjMzNi0wMzljMjBjMC1hNTI2LTRhOWEtYTc4Ni0yMmYwMTc1OTRiMDQiLCJzdV9pbnN0YW5jZSI6ZmFsc2UsInRydXN0ZWVfZW1haWwiOiJuLm1hbWFldkBpc3BzeXN0ZW0uY29tIiwidHJ1c3RlZV9pZCI6NCwieHNyZl90b2tlbiI6IjMzNi0wMzljMjBjMC1hNTI2LTRhOWEtYTc4Ni0yMmYwMTc1OTRiMDQifQ==
  74. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Got header: [x-forwarded-for]172.19.0.12
  75. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Got header: [x-remote-ip]172.19.0.12
  76. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Got header: [x-request-id]a78798d39ec883f8ce3cccd39a58b983
  77. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12] Got header: [x-xsrf-token]336-039c20c0-a526-4a9a-a786-22f017594b04
  78. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] RequestBody: {"cluster":22,"comment":"","host_filter":[],"host_limit":-1,"host_max":null,"ip":"172.31.36.34","lvm_vg_rename":false,"name":"HA_Node_1","network_autosetup_disabled":false,"overselling":1,"password":"*","ssh_port":22,"vcpu_limit":-1}
  79. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] End of POST request
  80. vm_1_writer.log:2023-04-10 06:26:12.271 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Invoke POST request callback
  81. vm_1_writer.log:2023-04-10 06:26:12.279 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT value,name,id FROM vm_bill_options WHERE name='license'' duration=373
  82. vm_1_writer.log:2023-04-10 06:26:12.280 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT vm5_data,spice_enabled,node_os_family,vxlan_mode,node_network_gateway,node_network_timeout,ha_error,ha_cluster_id,ha_agent_version,ha_checking_host,ha_enabled,net_bandwidth_mbitps,dns_servers,host_per_node_limit,proxy_enabled,domain_change_allowed,manage_disk_enabled,iso_enabled,cpu_overselling,overselling,qemu_version,domain_template,image_storage_path,os_storage_path,interface_count,imported_from,host_filter,host_distribution_policy,datacenter_type,virtualization_type,time_zone,comment,name,id FROM vm_cluster WHERE id='22'' duration=320
  83. vm_1_writer.log:2023-04-10 06:26:12.280 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT vm_storage2cluster.id AS id FROM vm_storage2cluster WHERE (vm_storage2cluster.cluster='22')' duration=175
  84. vm_1_writer.log:2023-04-10 06:26:12.281 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 611
  85. vm_1_writer.log:2023-04-10 06:26:12.281 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 611. Execute command: 'ssh-keygen -R 172.31.36.34'
  86. vm_1_writer.log:2023-04-10 06:26:12.285 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 611. Failed with status 65280
  87. vm_1_writer.log:2023-04-10 06:26:12.285 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 611. Output:
  88. vm_1_writer.log:2023-04-10 06:26:12.285 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 611. Error:
  89. vm_1_writer.log:2023-04-10 06:26:12.285 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 612
  90. vm_1_writer.log:2023-04-10 06:26:12.285 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 612. Execute command: '/usr/bin/sshpass -e -- ssh-copy-id -p 22 -o StrictHostKeyChecking=no -o ConnectTimeout=15 -i /opt/ispsystem/vm/etc/.ssh/vmmgr.1.pub root@172.31.36.34'
  91. vm_1_writer.log:2023-04-10 06:26:12.454 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 612. Finished successefully with status 0
  92. vm_1_writer.log:2023-04-10 06:26:12.454 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 612. Output:
  93. vm_1_writer.log:2023-04-10 06:26:12.454 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 612. Error:
  94. vm_1_writer.log:2023-04-10 06:26:12.455 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 667
  95. vm_1_writer.log:2023-04-10 06:26:12.455 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 667. Execute command: 'scripts/kvm/node_info.py --private-key /opt/ispsystem/vm/etc/.ssh/vmmgr.1 --host 172.31.36.34 --port 22'
  96. vm_1_writer.log:2023-04-10 06:26:12.969 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 667. Finished successefully with status 0
  97. vm_1_writer.log:2023-04-10 06:26:12.970 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 667. Output: {"os": {"version": "Astra Linux 1.7_x86-64", "name": "astra", "major_version": "1.7"}, "kvm_support": true, "kvm_enabled": true, "cpu_core_count": 5, "mem_mib": 3925, "hostname": "nmamaev.lxd.node.1.vm6-de-dev.com", "iface": "eth0"}
  98. vm_1_writer.log:2023-04-10 06:26:12.970 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 667. Error:
  99. vm_1_writer.log:2023-04-10 06:26:12.970 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Node Info: 0
  100. vm_1_writer.log:2023-04-10 06:26:12.970 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Node Info: {"cpu_core_count":5,"hostname":"nmamaev.lxd.node.1.vm6-de-dev.com","iface":"eth0","kvm_enabled":true,"kvm_support":true,"mem_mib":3925,"os":{"major_version":"1.7","name":"astra","version":"Astra Linux 1.7_x86-64"}}
  101. vm_1_writer.log:2023-04-10 06:26:12.970 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT name FROM vm_node WHERE hostname='nmamaev.lxd.node.1.vm6-de-dev.com' AND cluster='22' LIMIT 1' duration=469
  102. vm_1_writer.log:2023-04-10 06:26:12.971 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT id, cpu_number, additional_data FROM vm_node' duration=206
  103. vm_1_writer.log:2023-04-10 06:26:12.973 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT COUNT(id) FROM vm_node WHERE cluster = '22'' duration=166
  104. vm_1_writer.log:2023-04-10 06:26:12.983 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='UPDATE vm_cluster SET node_os_family='debian' WHERE id=22' duration=9751
  105. vm_1_writer.log:2023-04-10 06:26:12.984 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT id FROM vm_node WHERE name = 'HA_Node_1' LIMIT 1' duration=583
  106. vm_1_writer.log:2023-04-10 06:26:12.984 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT id FROM vm_node WHERE ip_addr = '172.31.36.34' LIMIT 1' duration=122
  107. vm_1_writer.log:2023-04-10 06:26:13.003 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='INSERT INTO vm_node (id,name,comment,cluster,ram_mib,max_vcpu_number,cpu_number,ip_addr,ssh_port,state,mon_install_date,os_version,overselling,hostname,host_filter) VALUES(632,'HA_Node_1','',22,3925,5,5,'172.31.36.34',22,'prepare_node','2023-04-10 06:26:12','Astra Linux 1.7_x86-64',1.000000,'nmamaev.lxd.node.1.vm6-de-dev.com','[]')' duration=19334
  108. vm_1_writer.log:2023-04-10 06:26:13.005 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT MAX(create_time) FROM vm_node_history WHERE event_type='field:state' AND ref='632'' duration=826
  109. vm_1_writer.log:2023-04-10 06:26:13.005 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='INSERT INTO vm_node_history (id,event_type,event_info,request_user,request_owner,request_ip,request_id,ref) VALUES (97,'field:state','{\"new_value\":\"prepare_node\",\"old_value\":null}','n.mamaev@ispsystem.com','---','172.19.0.12','a78798d39ec883f8ce3cccd39a58b983',632)' duration=362
  110. vm_1_writer.log:2023-04-10 06:26:13.006 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT date_create,old_values,take_slave_ip,vlan,is_main_network,is_default,state,gateway_v6,gateway_v4,ipv6,ipv4,node,name,id FROM vm_node_bridge WHERE vm_node_bridge.is_default AND vm_node_bridge.is_main_network='1' AND vm_node_bridge.node='632'' duration=158
  111. vm_1_writer.log:2023-04-10 06:26:13.006 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='INSERT INTO vm_node_bridge (id,name,node,state,is_default,take_slave_ip) VALUES(639,'vmbr0',632,'vm6_created','1','1')' duration=628
  112. vm_1_writer.log:2023-04-10 06:26:13.007 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT s.name, s.script, s.type, s.priority FROM vm_node_script s LEFT JOIN vm_node_script2cluster n_s2c ON n_s2c.script=s.id WHERE n_s2c.cluster='22'' duration=274
  113. vm_1_writer.log:2023-04-10 06:26:13.008 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT value,name,id FROM vm_settings WHERE name='task_timeout'' duration=145
  114. vm_1_writer.log:2023-04-10 06:26:13.008 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT value,name,id FROM vm_settings WHERE name='consul_task_ttl_in_sec'' duration=114
  115. vm_1_writer.log:2023-04-10 06:26:13.015 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT ha_cluster_id,ha_id,password_hidden,defer_reason,request_info,node,main_table_id,main_table,output,date_end,date_start,date_create,status,message_params,param,name,consul_id,id FROM vm_task_log WHERE consul_id='29642' AND name='node_create'' duration=324
  116. vm_1_writer.log:2023-04-10 06:26:13.019 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='INSERT INTO vm_task_log (id,consul_id,name,param,message_params,status,main_table,main_table_id,node,request_info) VALUES(129074,29642,'node_create','[\"--action\",\"node_create\",\"--virt-type\",\"kvm\",\"--params\",\"{\\\"autorun_scripts\\\":[],\\\"cluster_id\\\":22,\\\"datacenter_type\\\":\\\"common\\\",\\\"id\\\":632,\\\"image_storage_path\\\":\\\"/image\\\",\\\"ip\\\":\\\"172.31.36.34\\\",\\\"network_autosetup_disabled\\\":false,\\\"node_addr_v6\\\":null,\\\"node_interface_name\\\":\\\"eth0\\\",\\\"os_major_version\\\":\\\"1.7\\\",\\\"os_name\\\":\\\"astra\\\",\\\"os_storage_path\\\":\\\"/share\\\",\\\"os_version\\\":\\\"Astra Linux 1.7_x86-64\\\",\\\"proxy_enabled\\\":false,\\\"spice_enabled\\\":true,\\\"ssh_port\\\":22,\\\"timezone\\\":\\\"UTC\\\"}\"]','{\"ip_addr\":\"172.31.36.34\",\"name\":\"HA_Node_1\"}','created','vm_node',632,632,'{\"callback_counter\":16,\"first_start\":1817892135276426,\"need_log\":true,\"request\":{\"extra_tags\":[\"batch.0c1c5516-8883-f662-489c-821bd04a167a\"],\"id\":\"a78798d39ec883f8ce3cccd39a58b983\",\"ip\":\"172.19.0.12\",\"user\":{\"id\":4,\"name\":\"n.mamaev@ispsystem.com\"}},\"runtime\":740934500}')' duration=3486
  117. vm_1_writer.log:2023-04-10 06:26:13.021 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT vm_storage2cluster.storage AS storage FROM vm_storage2cluster WHERE (vm_storage2cluster.cluster='22')' duration=199
  118. vm_1_writer.log:2023-04-10 06:26:13.021 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT imported_from,vm5_data,is_network,storage_params,state,type,comment,name,id FROM vm_storage WHERE id='509'' duration=409
  119. vm_1_writer.log:2023-04-10 06:26:13.023 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT hdd_overselling,ha_params,params,pool_name,enabled,is_main,cluster,storage,id FROM vm_storage2cluster WHERE storage='509' AND cluster='22'' duration=247
  120. vm_1_writer.log:2023-04-10 06:26:13.024 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='INSERT INTO vm_storage2node (id,storage,node,state,hdd_overselling) VALUES(630,509,632,'creating',1.000000)' duration=1059
  121. vm_1_writer.log:2023-04-10 06:26:13.024 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Create notify 'instance/1/notify/create' with content '{"notifies":["storage/509/node/632"]}'
  122. vm_1_writer.log:2023-04-10 06:26:13.024 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT imported_from,vm5_data,is_network,storage_params,state,type,comment,name,id FROM vm_storage WHERE id='510'' duration=214
  123. vm_1_writer.log:2023-04-10 06:26:13.024 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT hdd_overselling,ha_params,params,pool_name,enabled,is_main,cluster,storage,id FROM vm_storage2cluster WHERE storage='510' AND cluster='22'' duration=205
  124. vm_1_writer.log:2023-04-10 06:26:13.025 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='INSERT INTO vm_storage2node (id,storage,node,state,hdd_overselling) VALUES(631,510,632,'creating',1.000000)' duration=562
  125. vm_1_writer.log:2023-04-10 06:26:13.025 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Create notify 'instance/1/notify/create' with content '{"notifies":["storage/510/node/632"]}'
  126. vm_1_writer.log:2023-04-10 06:26:13.025 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT value,name,id FROM vm_settings WHERE name='task_timeout'' duration=175
  127. vm_1_writer.log:2023-04-10 06:26:13.026 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT value,name,id FROM vm_settings WHERE name='consul_task_ttl_in_sec'' duration=148
  128. vm_1_writer.log:2023-04-10 06:26:13.040 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='SELECT ha_cluster_id,ha_id,password_hidden,defer_reason,request_info,node,main_table_id,main_table,output,date_end,date_start,date_create,status,message_params,param,name,consul_id,id FROM vm_task_log WHERE consul_id='29643' AND name='connect_storages'' duration=370
  129. vm_1_writer.log:2023-04-10 06:26:13.040 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='INSERT INTO vm_task_log (id,consul_id,name,param,message_params,status,main_table,main_table_id,request_info) VALUES(129075,29643,'connect_storages','[\"--action\",\"connect_storages\",\"--virt-type\",\"kvm\",\"--params\",\"{\\\"cluster_id\\\":22,\\\"node_id\\\":632,\\\"node_ip\\\":\\\"172.31.36.34\\\",\\\"node_ssh_port\\\":22,\\\"storages\\\":[{\\\"id\\\":509,\\\"type\\\":\\\"network_lvm\\\"},{\\\"id\\\":510,\\\"type\\\":\\\"network_lvm\\\"}]}\"]','{\"name\":\"HA_Node_1\"}','created','vm_node',632,'{\"callback_counter\":26,\"first_start\":1817892135276426,\"need_log\":true,\"request\":{\"extra_tags\":[\"batch.0c1c5516-8883-f662-489c-821bd04a167a\"],\"id\":\"a78798d39ec883f8ce3cccd39a58b983\",\"ip\":\"172.19.0.12\",\"user\":{\"id\":4,\"name\":\"n.mamaev@ispsystem.com\"}},\"runtime\":744565076}')' duration=712
  130. vm_1_writer.log:2023-04-10 06:26:13.047 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc001520 query='INSERT INTO vm_node_log (id,name,node,task,request_trustee,request_owner) VALUES(421,'node_create',632,129074,4,4)' duration=2541
  131. vm_1_writer.log:2023-04-10 06:26:13.047 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Create notify 'instance/1/notify/create' with content '{"notifies":["node_history/421","node/632/node_history/421"]}'
  132. vm_1_writer.log:2023-04-10 06:26:13.047 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Create notify 'instance/1/notify/update' with content '{"notifies":["cluster/22"]}'
  133. vm_1_writer.log:2023-04-10 06:26:13.047 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Create notify 'instance/1/notify/create' with content '{"notifies":["node/632"]}'
  134. vm_1_writer.log:2023-04-10 06:26:13.047 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Response body '{"id":632,"storages_task":29643,"task":29642}'
  135. vm_1_writer.log:2023-04-10 06:27:32.785 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] request method: POST url: /node/632 version: HTTP/1.0
  136. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Handler 'node/{node_id}' choosen
  137. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [connection]close
  138. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [content-length]43
  139. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [global-attributes]{"callback_counter":6,"first_start":1817893060963609,"request":{"id":"a78798d39ec883f8ce3cccd39a58b983","ip":"172.19.0.12","user":{"name":"n.mamaev@ispsystem.com"}},"runtime":114499506}
  140. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [host]instance-1
  141. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [instance-id]1
  142. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [internal-auth]eyJyb2xlcyI6WyJAaW50ZXJuYWwiXX0=
  143. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [x-forwarded-for]172.19.0.12
  144. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [x-remote-ip]172.19.0.12
  145. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [x-request-id]ec6aeb2b5aec3446027be7c42cf9a0f1
  146. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] RequestBody: {"values":{"state":"setup_virtualization"}}
  147. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] End of POST request
  148. vm_1_writer.log:2023-04-10 06:27:32.786 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Invoke POST request callback
  149. vm_1_writer.log:2023-04-10 06:27:32.788 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc03c2d0 query='SELECT additional_data,vm5_data,update_dc_networks_failed,network_error,network_settings_lock,ha_error,ha_disable_failed,ha_state,host_filter,host_limit,selinux_enforcing,hostname,overselling,lxd_version,qemu_version,libvirt_version,uptime,kernel_version,os_version,metrics_timestamp,mon_install_date,mon_installed,host_creation_blocked,property,disk_params,gray_ips,problems,spice_error,bird_error,lxd_error,libvirt_error,frr_error,connection_error,state,ssh_port,vnc_addr,ip_addr_v6,ip_addr,socket_number,vcpu_limit,cpu_number,max_vcpu_number,ram_mib,cluster,comment,name,id FROM vm_node WHERE id='632'' duration=562
  150. vm_1_writer.log:2023-04-10 06:27:32.788 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc03c2d0 query='SELECT vm5_data,spice_enabled,node_os_family,vxlan_mode,node_network_gateway,node_network_timeout,ha_error,ha_cluster_id,ha_agent_version,ha_checking_host,ha_enabled,net_bandwidth_mbitps,dns_servers,host_per_node_limit,proxy_enabled,domain_change_allowed,manage_disk_enabled,iso_enabled,cpu_overselling,overselling,qemu_version,domain_template,image_storage_path,os_storage_path,interface_count,imported_from,host_filter,host_distribution_policy,datacenter_type,virtualization_type,time_zone,comment,name,id FROM vm_cluster WHERE id='22'' duration=244
  151. vm_1_writer.log:2023-04-10 06:27:32.790 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc03c2d0 query='UPDATE vm_node SET state='setup_virtualization' WHERE id=632' duration=2161
  152. vm_1_writer.log:2023-04-10 06:27:32.791 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc03c2d0 query='SELECT MAX(create_time) FROM vm_node_history WHERE event_type='field:state' AND ref='632'' duration=215
  153. vm_1_writer.log:2023-04-10 06:27:32.791 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc03c2d0 query='INSERT INTO vm_node_history (id,prev_time,event_type,event_info,request_user,request_owner,request_ip,request_id,ref) VALUES (100,'2023-04-10 06:26:13','field:state','{\"new_value\":\"setup_virtualization\",\"old_value\":\"setup_virtualization\"}','n.mamaev@ispsystem.com','---','172.19.0.12','a78798d39ec883f8ce3cccd39a58b983',632)' duration=234
  154. vm_1_writer.log:2023-04-10 06:27:32.792 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Create notify 'instance/1/notify/update' with content '{"notifies":["node/632"]}'
  155. vm_1_writer.log:2023-04-10 06:27:32.792 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Response body '{"id":632}'
  156. vm_1_writer.log:2023-04-10 06:28:56.034 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] request method: POST url: /node/632 version: HTTP/1.0
  157. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Handler 'node/{node_id}' choosen
  158. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [connection]close
  159. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [content-length]60
  160. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [global-attributes]{"callback_counter":12,"first_start":1817893060963609,"request":{"id":"a78798d39ec883f8ce3cccd39a58b983","ip":"172.19.0.12","user":{"name":"n.mamaev@ispsystem.com"}},"runtime":79710276259}
  161. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [host]instance-1
  162. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [instance-id]1
  163. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [internal-auth]eyJyb2xlcyI6WyJAaW50ZXJuYWwiXX0=
  164. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [x-forwarded-for]172.19.0.12
  165. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [x-remote-ip]172.19.0.12
  166. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [x-request-id]395a8b497c15461216df400b73c695ac
  167. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] RequestBody: {"values":{"max_vcpu_number":255,"vnc_addr":"172.31.36.34"}}
  168. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] End of POST request
  169. vm_1_writer.log:2023-04-10 06:28:56.035 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Invoke POST request callback
  170. vm_1_writer.log:2023-04-10 06:28:56.037 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc05e0f0 query='SELECT additional_data,vm5_data,update_dc_networks_failed,network_error,network_settings_lock,ha_error,ha_disable_failed,ha_state,host_filter,host_limit,selinux_enforcing,hostname,overselling,lxd_version,qemu_version,libvirt_version,uptime,kernel_version,os_version,metrics_timestamp,mon_install_date,mon_installed,host_creation_blocked,property,disk_params,gray_ips,problems,spice_error,bird_error,lxd_error,libvirt_error,frr_error,connection_error,state,ssh_port,vnc_addr,ip_addr_v6,ip_addr,socket_number,vcpu_limit,cpu_number,max_vcpu_number,ram_mib,cluster,comment,name,id FROM vm_node WHERE id='632'' duration=514
  171. vm_1_writer.log:2023-04-10 06:28:56.037 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc05e0f0 query='SELECT vm5_data,spice_enabled,node_os_family,vxlan_mode,node_network_gateway,node_network_timeout,ha_error,ha_cluster_id,ha_agent_version,ha_checking_host,ha_enabled,net_bandwidth_mbitps,dns_servers,host_per_node_limit,proxy_enabled,domain_change_allowed,manage_disk_enabled,iso_enabled,cpu_overselling,overselling,qemu_version,domain_template,image_storage_path,os_storage_path,interface_count,imported_from,host_filter,host_distribution_policy,datacenter_type,virtualization_type,time_zone,comment,name,id FROM vm_cluster WHERE id='22'' duration=385
  172. vm_1_writer.log:2023-04-10 06:28:56.040 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc05e0f0 query='UPDATE vm_node SET max_vcpu_number=255,vnc_addr='172.31.36.34' WHERE id=632' duration=2033
  173. vm_1_writer.log:2023-04-10 06:28:56.041 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Create notify 'instance/1/notify/update' with content '{"notifies":["node/632"]}'
  174. vm_1_writer.log:2023-04-10 06:28:56.041 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Response body '{"id":632}'
  175. vm_1_writer.log:2023-04-10 06:28:56.051 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] request method: POST url: /node/632 version: HTTP/1.0
  176. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Handler 'node/{node_id}' choosen
  177. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [connection]close
  178. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [content-length]29
  179. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [global-attributes]{"callback_counter":24,"first_start":1817893060963609,"request":{"id":"a78798d39ec883f8ce3cccd39a58b983","ip":"172.19.0.12","user":{"name":"n.mamaev@ispsystem.com"}},"runtime":162960517051}
  180. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [host]instance-1
  181. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [instance-id]1
  182. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [internal-auth]eyJyb2xlcyI6WyJAaW50ZXJuYWwiXX0=
  183. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [x-forwarded-for]172.19.0.12
  184. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [x-remote-ip]172.19.0.12
  185. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got header: [x-request-id]f83a7c719cd47cae828916b6135abd5a
  186. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] RequestBody: {"values":{"state":"active"}}
  187. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] End of POST request
  188. vm_1_writer.log:2023-04-10 06:28:56.052 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Invoke POST request callback
  189. vm_1_writer.log:2023-04-10 06:28:56.055 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc03ee00 query='SELECT additional_data,vm5_data,update_dc_networks_failed,network_error,network_settings_lock,ha_error,ha_disable_failed,ha_state,host_filter,host_limit,selinux_enforcing,hostname,overselling,lxd_version,qemu_version,libvirt_version,uptime,kernel_version,os_version,metrics_timestamp,mon_install_date,mon_installed,host_creation_blocked,property,disk_params,gray_ips,problems,spice_error,bird_error,lxd_error,libvirt_error,frr_error,connection_error,state,ssh_port,vnc_addr,ip_addr_v6,ip_addr,socket_number,vcpu_limit,cpu_number,max_vcpu_number,ram_mib,cluster,comment,name,id FROM vm_node WHERE id='632'' duration=290
  190. vm_1_writer.log:2023-04-10 06:28:56.055 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc03ee00 query='SELECT vm5_data,spice_enabled,node_os_family,vxlan_mode,node_network_gateway,node_network_timeout,ha_error,ha_cluster_id,ha_agent_version,ha_checking_host,ha_enabled,net_bandwidth_mbitps,dns_servers,host_per_node_limit,proxy_enabled,domain_change_allowed,manage_disk_enabled,iso_enabled,cpu_overselling,overselling,qemu_version,domain_template,image_storage_path,os_storage_path,interface_count,imported_from,host_filter,host_distribution_policy,datacenter_type,virtualization_type,time_zone,comment,name,id FROM vm_cluster WHERE id='22'' duration=271
  191. vm_1_writer.log:2023-04-10 06:28:56.058 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc03ee00 query='UPDATE vm_node SET state='active' WHERE id=632' duration=2374
  192. vm_1_writer.log:2023-04-10 06:28:56.058 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc03ee00 query='SELECT MAX(create_time) FROM vm_node_history WHERE event_type='field:state' AND ref='632'' duration=238
  193. vm_1_writer.log:2023-04-10 06:28:56.059 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] db: id=0x7e68cc03ee00 query='INSERT INTO vm_node_history (id,prev_time,event_type,event_info,request_user,request_owner,request_ip,request_id,ref) VALUES (103,'2023-04-10 06:27:32','field:state','{\"new_value\":\"active\",\"old_value\":\"active\"}','n.mamaev@ispsystem.com','---','172.19.0.12','a78798d39ec883f8ce3cccd39a58b983',632)' duration=210
  194. vm_1_writer.log:2023-04-10 06:28:56.060 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Create notify 'instance/1/notify/update' with content '{"notifies":["node/632"]}'
  195. vm_1_writer.log:2023-04-10 06:28:56.060 [pid=565] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Response body '{"id":632}'
  196. vmctl.log:2023-04-10 06:26:13.085 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got action node_create with params
  197. vmctl.log:2023-04-10 06:26:13.085 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Using VmctlMode::VMCTL virt-type kvm
  198. vmctl.log:2023-04-10 06:26:13.088 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Stdin params: {"before_execute":{"last_notify":5431268},"stdin":{}}
  199. vmctl.log:2023-04-10 06:26:13.089 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Executing action node_create with params
  200. vmctl.log:2023-04-10 06:26:13.089 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] InternalCall service: nodewatch path: get_node_info
  201. vmctl.log:2023-04-10 06:26:13.197 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] NODE_STAGE: certificate
  202. vmctl.log:2023-04-10 06:26:13.198 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] try to create private key
  203. vmctl.log:2023-04-10 06:26:13.198 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 676
  204. vmctl.log:2023-04-10 06:26:13.198 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 676. Execute command: '/usr/bin/certtool --generate-privkey > /tmp/aa63-4dd5-d39c-7110-key.pem'
  205. vmctl.log:2023-04-10 06:26:13.448 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 676. Finished successefully with status 0
  206. vmctl.log:2023-04-10 06:26:13.448 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 676. Output:
  207. vmctl.log:2023-04-10 06:26:13.448 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 676. Error: Generating a 3072 bit RSA private key...
  208. vmctl.log:2023-04-10 06:26:13.448 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] generated private key /tmp/aa63-4dd5-d39c-7110-key.pem
  209. vmctl.log:2023-04-10 06:26:13.448 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] try to create certificate
  210. vmctl.log:2023-04-10 06:26:13.448 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 678
  211. vmctl.log:2023-04-10 06:26:13.448 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 678. Execute command: '/usr/bin/certtool --generate-certificate --template a1c8e575d-template --load-privkey /tmp/aa63-4dd5-d39c-7110-key.pem --load-ca-certificate /etc/pki/CA/cacert.pem --load-ca-privkey /opt/ispsystem/vm/etc/vmmgr.pem --outfile /tmp/181b-44af-84c8-0fda'
  212. vmctl.log:2023-04-10 06:26:13.456 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 678. Finished successefully with status 0
  213. vmctl.log:2023-04-10 06:26:13.456 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 678. Output:
  214. vmctl.log:2023-04-10 06:26:13.456 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 678. Error: Generating a signed certificate...
  215. vmctl.log:2023-04-10 06:26:13.456 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] generated country=XX
  216. vmctl.log:2023-04-10 06:26:13.456 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] try to create private key
  217. vmctl.log:2023-04-10 06:26:13.456 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 679
  218. vmctl.log:2023-04-10 06:26:13.456 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 679. Execute command: '/usr/bin/openssl req -newkey rsa:3072 -nodes -keyout /tmp/9463-ee1d-9187-7785-key.pem -out /etc/pki/libvirt/private/client.csr -subj /C=XX/ST=XX/L=XX/O=XX/OU=XX/CN=node@vmmanager'
  219. vmctl.log:2023-04-10 06:26:13.563 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 679. Finished successefully with status 0
  220. vmctl.log:2023-04-10 06:26:13.563 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 679. Output:
  221. vmctl.log:2023-04-10 06:26:13.563 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 679. Error: Generating a RSA private key
  222. vmctl.log:2023-04-10 06:26:13.563 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] generated private key /tmp/9463-ee1d-9187-7785-key.pem
  223. vmctl.log:2023-04-10 06:26:13.563 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] try to create certificate
  224. vmctl.log:2023-04-10 06:26:13.563 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 680
  225. vmctl.log:2023-04-10 06:26:13.563 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 680. Execute command: '/usr/bin/openssl x509 -req -in /etc/pki/libvirt/private/client.csr -CAkey /opt/ispsystem/vm/etc/vmmgr.pem -CA /etc/pki/CA/cacert.pem -CAcreateserial -days 10950 -extfile openssl_conf -out /tmp/7e1c-e989-5c20-daf4'
  226. vmctl.log:2023-04-10 06:26:13.573 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 680. Finished successefully with status 0
  227. vmctl.log:2023-04-10 06:26:13.573 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 680. Output:
  228. vmctl.log:2023-04-10 06:26:13.573 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 680. Error: Signature ok
  229. vmctl.log:2023-04-10 06:26:13.573 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] generated country=XX
  230. vmctl.log:2023-04-10 06:26:13.573 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 681
  231. vmctl.log:2023-04-10 06:26:13.573 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 681. Execute command: '/usr/bin/ansible-playbook -i 172.31.36.34:22, -e targets=all -e ansible_python_interpreter='auto' -e ca_cert='/etc/pki/CA/cacert.pem' -e server_key='/tmp/aa63-4dd5-d39c-7110-key.pem' -e server_cert='/tmp/181b-44af-84c8-0fda' -e client_key='/tmp/9463-ee1d-9187-7785-key.pem' -e client_cert='/tmp/7e1c-e989-5c20-daf4' -e server_key_dest='/etc/pki/libvirt/private/serverkey.pem' -e server_cert_dest='/etc/pki/libvirt/servercert.pem' -e client_key_dest='/etc/pki/libvirt/private/clientkey.pem' -e client_cert_dest='/etc/pki/libvirt/clientcert.pem' etc/playbooks/node/certificate.yml --key-file /opt/ispsystem/vm/etc/.ssh/vmmgr.1 --timeout 60'
  232. vmctl.log:2023-04-10 06:26:18.158 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 681. Finished successefully with status 0
  233. vmctl.log:2023-04-10 06:26:18.158 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 681. Output:
  234. vmctl.log:2023-04-10 06:26:18.158 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 681. Error:
  235. vmctl.log:2023-04-10 06:26:18.158 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] process out:
  236. vmctl.log:2023-04-10 06:26:18.158 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] NODE_STAGE: prepare
  237. vmctl.log:2023-04-10 06:26:18.158 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 813
  238. vmctl.log:2023-04-10 06:26:18.158 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 813. Execute command: '/usr/bin/ansible-playbook -i 172.31.36.34:22, -e targets=all -e ansible_python_interpreter='auto' -e timezone='UTC' -e os_storage_path='/share' -e image_storage_path='/image' -e datacenter_type='common' -e is_lxd='0' etc/playbooks/node/prepare.yml --key-file /opt/ispsystem/vm/etc/.ssh/vmmgr.1 --timeout 60'
  239. vmctl.log:2023-04-10 06:27:10.197 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 813. Finished successefully with status 0
  240. vmctl.log:2023-04-10 06:27:10.197 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 813. Output:
  241. vmctl.log:2023-04-10 06:27:10.197 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 813. Error:
  242. vmctl.log:2023-04-10 06:27:10.197 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] process out:
  243. vmctl.log:2023-04-10 06:27:10.197 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] NODE_STAGE: firewall
  244. vmctl.log:2023-04-10 06:27:10.198 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 1589
  245. vmctl.log:2023-04-10 06:27:10.198 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 1589. Execute command: '/usr/bin/ansible-playbook -i 172.31.36.34:22, -e targets=all -e ansible_python_interpreter='auto' -e datacenter_type='common' -e ssh_port='22' -e network_autosetup_enabled='1' -e is_lxd='0' -e dc_ips='' -e dc_ips6='' etc/playbooks/node/firewall.yml --key-file /opt/ispsystem/vm/etc/.ssh/vmmgr.1 --timeout 60'
  246. vmctl.log:2023-04-10 06:27:32.784 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 1589. Finished successefully with status 0
  247. vmctl.log:2023-04-10 06:27:32.784 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 1589. Output:
  248. vmctl.log:2023-04-10 06:27:32.784 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 1589. Error:
  249. vmctl.log:2023-04-10 06:27:32.784 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] process out:
  250. vmctl.log:2023-04-10 06:27:32.784 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] InternalCall service: vm path: node/632
  251. vmctl.log:2023-04-10 06:27:32.793 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] NODE_STAGE: libvirtd
  252. vmctl.log:2023-04-10 06:27:32.793 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 2182
  253. vmctl.log:2023-04-10 06:27:32.793 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 2182. Execute command: '/usr/bin/ansible-playbook -i 172.31.36.34:22, -e targets=all -e ansible_python_interpreter='auto' -e node_ip='172.31.36.34' -e is_lxd='0' etc/playbooks/node/virt.yml --key-file /opt/ispsystem/vm/etc/.ssh/vmmgr.1 --timeout 60'
  254. vmctl.log:2023-04-10 06:28:52.740 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 2182. Finished successefully with status 0
  255. vmctl.log:2023-04-10 06:28:52.740 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 2182. Output:
  256. vmctl.log:2023-04-10 06:28:52.740 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 2182. Error:
  257. vmctl.log:2023-04-10 06:28:52.740 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] process out:
  258. vmctl.log:2023-04-10 06:28:52.740 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] NODE_STAGE: network
  259. vmctl.log:2023-04-10 06:28:52.741 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 3274
  260. vmctl.log:2023-04-10 06:28:52.741 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3274. Execute command: 'scripts/kvm/node_network_astra_1.7.py --node-address 172.31.36.34 --port 22 --private-key /opt/ispsystem/vm/etc/.ssh/vmmgr.1 --datacenter_type common'
  261. vmctl.log:2023-04-10 06:28:53.496 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3274. Finished successefully with status 0
  262. vmctl.log:2023-04-10 06:28:53.496 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3274. Output:
  263. vmctl.log:2023-04-10 06:28:53.496 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3274. Error:
  264. vmctl.log:2023-04-10 06:28:53.496 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] process out:
  265. vmctl.log:2023-04-10 06:28:53.496 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] NODE_STAGE: virt_bridge
  266. vmctl.log:2023-04-10 06:28:53.496 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 3278
  267. vmctl.log:2023-04-10 06:28:53.496 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3278. Execute command: 'scripts/kvm/node_virt_bridge.py --node_addr 172.31.36.34'
  268. vmctl.log:2023-04-10 06:28:53.718 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3278. Finished successefully with status 0
  269. vmctl.log:2023-04-10 06:28:53.718 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3278. Output: {}
  270. vmctl.log:2023-04-10 06:28:53.718 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3278. Error:
  271. vmctl.log:2023-04-10 06:28:53.718 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] process out: {}
  272. vmctl.log:2023-04-10 06:28:53.719 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 3283
  273. vmctl.log:2023-04-10 06:28:53.719 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3283. Execute command: 'scripts/kvm/node_vnc_address.py --node-address 172.31.36.34 --port 22 --private-key /opt/ispsystem/vm/etc/.ssh/vmmgr.1'
  274. vmctl.log:2023-04-10 06:28:53.984 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3283. Finished successefully with status 0
  275. vmctl.log:2023-04-10 06:28:53.984 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3283. Output: {"vnc_addr": "172.31.36.34"}
  276. vmctl.log:2023-04-10 06:28:53.984 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3283. Error:
  277. vmctl.log:2023-04-10 06:28:53.984 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] process out: {"vnc_addr": "172.31.36.34"}
  278. vmctl.log:2023-04-10 06:28:53.985 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 3286
  279. vmctl.log:2023-04-10 06:28:53.985 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3286. Execute command: 'scripts/kvm/node_configure_spice.py --node-address 172.31.36.34 --node-port 22 --private-key /opt/ispsystem/vm/etc/.ssh/vmmgr.1'
  280. vmctl.log:2023-04-10 06:28:54.502 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3286. Finished successefully with status 0
  281. vmctl.log:2023-04-10 06:28:54.502 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3286. Output: {"spice_enabled": true}
  282. vmctl.log:2023-04-10 06:28:54.502 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3286. Error:
  283. vmctl.log:2023-04-10 06:28:54.502 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] process out: {"spice_enabled": true}
  284. vmctl.log:2023-04-10 06:28:54.502 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] NODE_STAGE: info
  285. vmctl.log:2023-04-10 06:28:54.503 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 3289
  286. vmctl.log:2023-04-10 06:28:54.503 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3289. Execute command: '/usr/bin/ansible-playbook -i 172.31.36.34:22, -e targets=all -e ansible_python_interpreter='auto' etc/playbooks/node/info.yml --key-file /opt/ispsystem/vm/etc/.ssh/vmmgr.1 --timeout 60'
  287. vmctl.log:2023-04-10 06:28:56.032 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3289. Finished successefully with status 0
  288. vmctl.log:2023-04-10 06:28:56.032 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3289. Output: {
  289. vmctl.log:2023-04-10 06:28:56.032 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3289. Error:
  290. vmctl.log:2023-04-10 06:28:56.032 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] process out: {
  291. vmctl.log:2023-04-10 06:28:56.033 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] InternalCall service: vm path: node/632
  292. vmctl.log:2023-04-10 06:28:56.041 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] InternalCall service: nodewatch path: update_nodes
  293. vmctl.log:2023-04-10 06:28:56.045 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] NODE_STAGE: autorun scripts
  294. vmctl.log:2023-04-10 06:28:56.045 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] InternalCall service: vm path: node/632
  295. vmctl.log:2023-04-10 06:28:56.061 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] run vmwatch for node 172.31.36.34
  296. vmctl.log:2023-04-10 06:28:56.061 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] InternalCall service: vmwatch path: watcher
  297. vmctl.log:2023-04-10 06:28:56.064 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] run ifacewatch for node 172.31.36.34
  298. vmctl.log:2023-04-10 06:28:56.064 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] InternalCall service: ifacewatch path: watcher
  299. vmctl.log:2023-04-10 06:28:56.070 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Registration in the statistics service '1.node.632'
  300. vmctl.log:2023-04-10 06:28:56.070 [pid=673] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] InternalCall service: statistic path: target
  301. vmctl.log:2023-04-10 06:28:56.164 [pid=3311] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Got action connect_storages with params
  302. vmctl.log:2023-04-10 06:28:56.164 [pid=3311] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Using VmctlMode::VMCTL virt-type kvm
  303. vmctl.log:2023-04-10 06:28:56.165 [pid=3311] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Stdin params: {"before_execute":{"last_notify":5432041},"stdin":{}}
  304. vmctl.log:2023-04-10 06:28:56.165 [pid=3311] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Executing action connect_storages with params
  305. vmctl.log:2023-04-10 06:28:56.165 [pid=3311] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] Fork. New Process PID: 3313
  306. vmctl.log:2023-04-10 06:28:56.166 [pid=3311] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3313. Execute command: 'scripts/kvm/connect_storages.py --node-address 172.31.36.34 --port 22 --private-key /opt/ispsystem/vm/etc/.ssh/vmmgr.1'
  307. vmctl.log:2023-04-10 06:28:57.348 [pid=3311] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3313. Finished successefully with status 0
  308. vmctl.log:2023-04-10 06:28:57.348 [pid=3311] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3313. Output: [{"id": 509, "error": "Command exited with message   A volume group called Network_lvm_dev_sda already exists.\n."}, {"id": 510, "error": "physical vollume \"/dev/sdb\" is busy, unknown volume group \"Network_lvm_dev_sda\" was foun"}]
  309. vmctl.log:2023-04-10 06:28:57.348 [pid=3311] [#a78798d39ec883f8ce3cccd39a58b983/172.19.0.12/@n.mamaev@ispsystem.com] PID: 3313. Error:
  310. vmwatch.log:2023/04/10 06:28:56 [pid=176] Web New request: "instance-1" - "POST" - "/watcher" - http.Header{"Connection":[]string{"close"}, "Content-Length":[]string{"58"}, "Global-Attributes":[]string{"{\"callback_counter\":30,\"first_start\":1817893060963609,\"request\":{\"id\":\"a78798d39ec883f8ce3cccd39a58b983\",\"ip\":\"172.19.0.12\",\"user\":{\"name\":\"n.mamaev@ispsystem.com\"}},\"runtime\":162978456407}"}, "Instance-Id":[]string{"1"}, "Internal-Auth":[]string{"on"}, "X-Forwarded-For":[]string{"172.19.0.12"}, "X-Remote-Ip":[]string{"172.19.0.12"}, "X-Request-Id":[]string{"e9f5ed5149e77716c6b958592b4d7d22"}}
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!