--- Log opened vr feb 10 00:00:34 2012 00:09 -!- Netsplit *.net <-> *.split quits: itamarh 00:09 -!- Netsplit over, joins: itamarh 01:19 -!- abaron_wfh: has quit [Ping timeout: 245 seconds] 02:02 -!- simon3z: has quit [Quit: Ex-Chat] 02:44 -!- jarod1: has joined #vdsm 03:19 -!- jbrooks: has quit [Quit: Ex-Chat] 03:30 -!- tdfkaSaggi: has joined #vdsm 03:38 -!- tdfkaSaggi: has quit [Read error: Connection reset by peer] 03:38 -!- tdfkaSaggi1: has joined #vdsm 04:19 -!- osier: has joined #vdsm 05:02 -!- Humble: has joined #vdsm 05:52 -!- deepakcs: has joined #vdsm 06:53 -!- itamarh1: has joined #vdsm 06:53 -!- itamarh: has quit [Ping timeout: 248 seconds] 07:39 -!- ykaul: has joined #vdsm 07:52 -!- xTs_clone: has joined #vdsm 07:56 -!- xTs_w: has quit [Ping timeout: 276 seconds] 08:02 -!- Netsplit *.net <-> *.split quits: deepakcs 08:07 -!- Netsplit over, joins: deepakcs 08:11 < deepakcs> tdfkaSaggi1, Hi 08:16 -!- lmh: has joined #vdsm 08:16 -!- lmh_: has quit [Ping timeout: 260 seconds] 08:21 -!- ykaul1: has joined #vdsm 08:23 -!- osier_: has joined #vdsm 08:26 -!- ykaul: has quit [Ping timeout: 252 seconds] 08:26 -!- osier: has quit [Ping timeout: 252 seconds] 08:47 -!- xTsssss: has joined #vdsm 08:50 -!- xTs_clone: has quit [Ping timeout: 276 seconds] 08:50 -!- tdfkaSaggi1: has quit [Ping timeout: 276 seconds] 08:50 -!- tdfkaSaggi: has joined #vdsm 09:04 -!- lmh: has quit [Ping timeout: 240 seconds] 09:06 -!- lmh: has joined #vdsm 09:29 -!- Netsplit *.net <-> *.split quits: dougsland 09:35 -!- dougsland: has joined #vdsm 09:42 -!- Netsplit *.net <-> *.split quits: dougsland 09:45 -!- dougsland: has joined #vdsm 09:45 -!- dougsland: has joined #vdsm 09:51 -!- Netsplit *.net <-> *.split quits: Yet_another_Bill 09:52 -!- Madkiss: has quit [Disconnected by services] 09:55 -!- Yet_another_Bill: has joined #vdsm 09:58 -!- simon3z: has joined #vdsm 10:21 -!- dougsland: has joined #vdsm 10:21 -!- dougsland: has joined #vdsm 10:28 -!- abaron_wfh: has joined #vdsm 10:32 -!- danpb: has joined #vdsm 10:41 < deepakcs> whats the latest vdsm version 3 or 3.1 ? 10:56 -!- dougsland: has joined #vdsm 10:56 -!- dougsland: has quit [Changing host] 10:56 -!- dougsland: has joined #vdsm 11:09 -!- xTsssss is now known as xTs_w 11:34 -!- jarod1: has quit [Remote host closed the connection] 11:38 -!- ykaul1: has quit [Quit: Leaving.] 12:21 < deepakcs> tdfkaSaggi, Hello 13:15 -!- SCarniel: has joined #vdsm 13:15 -!- SCarniel: has quit [Changing host] 13:15 -!- SCarniel: has joined #vdsm 13:19 -!- rmatinata: has quit [Ping timeout: 252 seconds] 13:25 -!- AlbertoSilva: has joined #vdsm 13:32 -!- AlbertoS_: has joined #vdsm 13:56 -!- rmatinata: has joined #vdsm 13:58 -!- danken: has joined #vdsm 13:58 -!- mode/#vdsm: by ChanServ 14:06 -!- osier_: has quit [Remote host closed the connection] 14:31 -!- acathrow: has joined #vdsm 14:39 -!- deepakcs: has quit [Quit: Leaving] 15:26 -!- abaron_wfh: has quit [Ping timeout: 252 seconds] 15:29 -!- dougsland: has joined #vdsm 15:31 -!- rharper: has joined #vdsm 16:13 < aglitke> Can anyone think of why I might be getting the error Exception: {'status': {'message': 'Not SPM', 'code': 654}} even when my connected storage pool shows 'spm_id': 1 ? 16:22 -!- ykaul: has joined #vdsm 17:08 -!- jbrooks: has joined #vdsm 17:35 -!- jbrooks: has quit [Remote host closed the connection] 17:38 -!- jbrooks: has joined #vdsm 17:53 -!- deepakcs: has joined #vdsm 17:57 < deepakcs> tdfkaSaggi, Hello, GM 18:00 < deepakcs> tdfkaSaggi, around ? 18:01 < tdfkaSaggi> deepakcs: ? 18:02 < deepakcs> tdfkaSaggi, Hi, got some Qs with regards to yest discusssion of ours. 18:02 < tdfkaSaggi> aglitke: is the state free? 18:03 < deepakcs> tdfkaSaggi, you had provided me with the PosixFsConnection features wiki link.. status shows Done, but i don't see any PosixFsConnection class in the vdsm code... 18:03 < tdfkaSaggi> aglitke: the spm_id field is not really what you expect it to be. I fixed it once but Engine is kind of depending on the old broken behavior so you can't really trust the response. 18:04 < aglitke> hmm. Can I trust the task result for startSPM? 18:04 < tdfkaSaggi> aglitke: yes 18:04 < tdfkaSaggi> the deal is 18:04 < tdfkaSaggi> if someone claims to be the SPM 18:04 < tdfkaSaggi> then he is 18:04 < aglitke> I just filed a bug to cover what I am seeing: https://bugzilla.redhat.com/show_bug.cgi?id=789381 18:04 < tdfkaSaggi> but if someone claims someone else is the SPM, there is a chance he's wrong 18:05 < tdfkaSaggi> tdfkaSaggi: Yea, it's called sharedfs now, I will hopefully have the patches for the new API submitted next week 18:05 < tdfkaSaggi> in any case just use "sharedfs" as the connection type 18:06 < deepakcs> tdfkaSaggi, the wiki link says ovirt gui part is not done.. is this what u intend to submit ? 18:06 < aglitke> tdfkaSaggi, Are you an expert in the vdsm threading/processpool code? 18:06 -!- aliguori: has quit [Remote host closed the connection] 18:06 < tdfkaSaggi> aglitke: yes, yes I am 18:07 < deepakcs> tdfkaSaggi, if gui part is not yet there, how can i select sharedfs ? 18:07 < aglitke> ok. I am nervous that cherrypy 18:07 < aglitke> cherrypy's threadpools might be interacting badly with vdsms threads/ 18:08 < tdfkaSaggi> deepakcs: vdsClient? I don't know as I said I don't use the GUI. I can't even seem to get it installed 18:08 < tdfkaSaggi> aglitke: how so 18:08 < deepakcs> tdfkaSaggi, ok, i never tried vdsclient, need to check. 18:08 < aglitke> I can create a storagedomain with the xmlrpc bindings, but when I try to create via rest I get hangs in some of the locking code. 18:09 -!- aliguori: has joined #vdsm 18:09 < deepakcs> tdfkaSaggi, I am lookng at vdsm-gluster integration aspect 18:09 < deepakcs> deepakcs: OK, then for now we will use the PoisxFsConnection to get it working for 3.1 18:09 < tdfkaSaggi> aglitke: Did I ever send you my gdbinit script? 18:09 < deepakcs> tdfkaSaggi, ^^ 3.1 you mean the ovirt version or vdsm version ? 18:09 < tdfkaSaggi> deepakcs: they will get it done for 3.1 18:09 < aglitke> tdfkaSaggi, no 18:10 < tdfkaSaggi> deepakcs: vdsm's schedule is a bit tighter then engine's 18:10 < deepakcs> tdfkaSaggi, 3.1 is ovirt version ? 18:10 < tdfkaSaggi> deepakcs: yes 18:10 < tdfkaSaggi> sorry RHEV version 18:11 < deepakcs> rhev-m or rhev-h, i use normal linux box as host, not ovirt node 18:11 < tdfkaSaggi> You will see all of it upstream soon enough 18:11 < deepakcs> tdfkaSaggi, so if i use a sharedfs connection, i will have to migrate the code to posixfsconn ? 18:12 < tdfkaSaggi> RHEV 3.1 is a product minor release. I will happend someday I'm not sure. Stuff appear on upstream sooner. You should talk to the GUI guys about when are they going to push those specific parts 18:12 < tdfkaSaggi> aglitke: I'll upload it somewhere, it's good for debugging deadlocks 18:13 < deepakcs> tdfkaSaggi, ok sure, will check on #ovirt... can you help answer my other q pls. I am a bit confused still. 18:13 < tdfkaSaggi> deepakcs: what other Q? 18:13 < aglitke> ok, cool. I am starting to wonder if I should just grab a process from the processpool and run cherrypy there. 18:14 < tdfkaSaggi> aglitke: no, you shouldn't 18:14 < aglitke> heh, ok. That's settled :) 18:14 < deepakcs> tdfkaSaggi, so if i use a sharedfs connection, i will have to migrate the code to posixfsconn ? 18:14 < tdfkaSaggi> aglitke: the process pool is going away once domain engines are in 18:14 -!- ykaul: has quit [Quit: Leaving.] 18:14 < aglitke> ahh 18:15 < tdfkaSaggi> deepakcs: The old API will be supported all through the 3.X releases of RHEV which is quite a long period. 18:16 < tdfkaSaggi> deepakcs: and the change to the new API is not very difficult. 18:16 < deepakcs> tdfkaSaggi, so if i have not started its a good idea to wait and then work over posixfsconnectin ? 18:16 -!- SCarniel: has quit [Quit: Saindo...] 18:16 < tdfkaSaggi> deepakcs: it will be in the next few days, you could also work on top my WIP patches in gerrit if you don't mind broken things 18:17 < tdfkaSaggi> aglitke: the process pool is a hack until we give each domain it's own process 18:17 < deepakcs> tdfkaSaggi, ok, i will wait then, i am still figuring out the details and charting out the design. 18:18 < aglitke> tdfkaSaggi, Ok. I will be happy to see some of that go away. It's pretty complex. 18:18 < deepakcs> tdfkaSaggi, one naive q... for vdsm-glsuter integration, who will do the gluster initialistion on the host, is it the admin who will setup glsuter on the host and then create storage domain via ovirt ? 18:19 < deepakcs> tdfkaSaggi, for eg: today the NFS share has to be setup prior to settign up NFS domain , so same would apply for any other storage type (glsuter here ) 18:19 -!- rmatinata: has quit [Ping timeout: 252 seconds] 18:20 < tdfkaSaggi> deepakcs: the plan is to have everything done VIA ovirt. But for phase 1 you will have to set up gluster yourself. 18:22 < deepakcs> tdfkaSaggi, ok, what is SDM ( I see it while reading this - http://www.ovirt.org/wiki/AddingGlusterSupportToOvirt ) 18:23 < tdfkaSaggi> aglitke: https://github.com/ficoos/gdb-python-scripts 18:23 < tdfkaSaggi> aglitke: I will add a readme one day but for now 18:24 < tdfkaSaggi> aglitke: pystack will give you the python stack trace 18:24 < tdfkaSaggi> aglitke: t a a pystack will print out the stack trace of all the threads 18:24 < aglitke> ahh, thanks! 18:24 < tdfkaSaggi> aglitke: Just look for two threads that are stuck in some sort of acquire 18:25 < tdfkaSaggi> aglitke: Since I wrote this debugging deadlocks is much much easier. you do need python-debuginfo installed 18:26 < aglitke> ok 18:26 -!- danpb: has quit [Quit: Leaving.] 18:26 < tdfkaSaggi> aglitke: feel free to add features. I don't have time to fiddle with this anymore but it has a lot of potential. 18:26 < aglitke> DO you happen to know if the underlying locking uses persistent objects (shmem, lockfiles,etc) that would need to be cleaned up if vdsm crashes? 18:27 < tdfkaSaggi> deepakcs: We are removing the storage pool concept so you will have only storage domains. So the SPM will become SDM where each host can be the manager of 1 or more domains 18:27 < tdfkaSaggi> aglitke: We use pthreads specifically. look at betterThreading.py for the nitty gritty details. 18:28 < aglitke> Yeah, only for Lock() and Condition() though 18:28 < tdfkaSaggi> everything is based on those 18:28 < aglitke> ok 18:28 < tdfkaSaggi> aglitke: we do some monkey patching so check if that isn't messing stuff up for you 18:29 < aglitke> ok 18:31 < tdfkaSaggi> anyway, t a a pystack will help you pinpoint the exact deadlock quite easily 18:31 < tdfkaSaggi> aglitke: ^^^ 18:31 < tdfkaSaggi> deepakcs: understood? 18:31 -!- rmatinata: has joined #vdsm 18:31 < aglitke> tdfkaSaggi, Thanks for your help. I'll run off and play with this a bit and allow you to return to your pile of work :) 18:32 < deepakcs> tdfkaSaggi, trying :) in SPM case, only one host was having SPM responsibility, getting adv of single MD writer, in SDM that will go away ? 18:32 < tdfkaSaggi> aglitke: You don't know the half of it. But I got a surprise on the way :) 18:32 < tdfkaSaggi> deepakcs: you still have 1 MD writer 18:32 < tdfkaSaggi> deepakcs: but the scope is a domain and not a pool 18:33 < deepakcs> tdfkaSaggi, ah ok.. so with SP gone... how do you intend to take care of the migration.. before migration was possible across domains of 1 pool, rite ? 18:34 < tdfkaSaggi> deepakcs: sanlock, there are already patches for cluster wide VM locking by Federico 18:38 < deepakcs> tdfkaSaggi, san here is the aggregation of all domains across hosts which are shared 18:39 < tdfkaSaggi> deepakcs: sanlock is a name of a project 18:40 < tdfkaSaggi> deepakcs: for cluster wide disk base locks 18:41 < deepakcs> tdfkaSaggi, ah ok... going ahead you mentioned that storage init can also be driven by ovirt.. will this be done via the vdsm 'services' framework - i read abt that services proposal somewhere 18:41 < tdfkaSaggi> deepakcs: http://libvirt.org/locking.html 18:41 < tdfkaSaggi> deepakcs: yes 18:41 * deepakcs looks up the link 18:43 < deepakcs> tdfkaSaggi, today the iso domain has to be NFS based only, when new storage types/domains will be introduced, will this be relaxed, meanign can i have ISO domain backed by gluster storage accessed via gluster client ? 18:44 < tdfkaSaggi> deepakcs: ISO domains will get a major re-haul, there are some suggestion for how it's going to go but nothing concrete. 18:45 < deepakcs> tdfkaSaggi, Oh... everything seems to be changing :) 18:45 < deepakcs> atleast on the storage side ;) 18:47 < tdfkaSaggi> deepakcs: Well the entire VM moved for being internal to libvirt 18:47 < tdfkaSaggi> and networking is getting some work done do 18:48 < deepakcs> tdfkaSaggi, but it should be possible to have ISO domain, storage/data domain to be on any of the storage types going forward... the 'has to be NFS' should go away.... is that the thinking or not ? 18:49 < tdfkaSaggi> deepakcs: We want to support as many storage backends as possible 18:49 < aglitke> tdfkaSaggi: So looks like a thread hangs in scanDomains for a long time. Is it okay for a cherrypy thread to call way down into tasks/worker code? 18:49 < tdfkaSaggi> aglitke: For what? 18:50 < aglitke> Basically, cherrypy calls into API.py directly. So its worker thread ends up calling deep into vdsm. 18:50 < aglitke> I assume that should be okay. 18:51 < tdfkaSaggi> aglitke: yea, as long as you don't bypass the API layer 18:51 < aglitke> Just odd that scanning localFS storage domains should take a minute and then time out. 18:51 < tdfkaSaggi> aglitke: If you have a stale NFS mount 18:51 < tdfkaSaggi> aglitke: it could get things stuck 18:51 < aglitke> not using any nfs mounts. Only localfs. 18:51 < tdfkaSaggi> check with mount 18:52 < aglitke> but the box does have an unrelated nfs share mounted. 18:52 < tdfkaSaggi> you might have forgotten to umount 18:53 < aglitke> Nothing suspicious there... Will keep looking. 18:53 < deepakcs> tdfkaSaggi, thanks, is there a plan to put all the thinking around these proposed and no-so-concrete changes on the wiki, or on the vdsm-devel ml ? 18:53 < deepakcs> so that folks like me can catch and work in line with the new changes for vdsm-gluster integration 18:54 < tdfkaSaggi> deepakcs: When I say we it's mostly me that's thinking far ahead. And in any case we prefer not to poke the bear before we have to. 18:56 -!- dougsland: has joined #vdsm 18:56 -!- dougsland: has joined #vdsm 18:58 < deepakcs> tdfkaSaggi, understand.. I was thinking to start a thread on vdsm-devel to discuss my thoughts on the vdsm-gluster integration , what do u think ? It could help others too 19:12 -!- deepakcs: has quit [Quit: Leaving] 19:33 -!- jbrooks: has quit [Remote host closed the connection] 19:34 -!- jbrooks: has joined #vdsm 20:11 -!- ykaul: has joined #vdsm 20:12 -!- simon3z: has quit [Quit: Ex-Chat] 20:58 -!- dougsland: has joined #vdsm 20:58 -!- dougsland: has joined #vdsm 21:37 -!- rmatinata: has quit [Ping timeout: 248 seconds] 22:09 -!- ykaul: has quit [Quit: Leaving.] 22:37 -!- rmatinata: has joined #vdsm 23:32 -!- danken: has quit [Quit: Leaving.] --- Log closed za feb 11 00:00:34 2012